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ABSTRACT 


This  Impact  Report  examines  the  current  status  of  performance  measurement  and 
capacity  planning  within  a  management-oriented,  conceptual  framework.  The 
current  and  future  importance  and  role  of  performance  measurement  and  capacity 
planning  for  MIS  organizations  are  described  and  assessed.  Detailed  recommendations 
are  provided  for  ensuring  effective  performance  measurement  and  capacity  planning. 
An  appendix  of  available  tools  is  also  provided. 
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I  INTRODUCTION 


INTRODUCTION 


Performance  measurement  and  capacity  planning  are  becoming  more 
important  to  MIS  management  as  well  as  to  general  management. 

This  subject  was  among  those  gathering  the  most  interest  at  INPUT'S 
1980  conference  of  MIS  executives. 

There  have  even  been  perceptive  references  in  the  general  business 
press  to  the  issues  involved  (e.g.,  a  December  1980  article  in  Business 
Week). 

Because  of  this  interest,  as  well  as  the  inherent  importance  of  the  subject  and 
recent  developments  which  strongly  affect  how  well  it  can  be  done,  INPUT  has 
devoted  this  Impact  Report  to  performance  measurement  and  capacity 
planning. 

INPUT  conducted  43  interviews  in  February  and  March  1981  to  ascertain: 

The  current  status  of  performance  measurement  and  capacity  planning. 
Issues  and  problems. 

Unmet  needs  and  likely  future  developments. 

Current  and  future  products  to  support  these  functions. 
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Appendices  A  through  C  contain  the  detailed  methodology  employed  and  the 
interview  guides  used. 

Many  of  the  data  are  presented  in  quantitative  form.  However,  qualitative 
judgments  based  upon  observations  made  in  the  course  of  the  research  have 
been  introduced  where  these  were  seen  to  be  important. 

Several  vendor  representatives  and  others  with  extensive  experience  kindly 
shared  their  experience  and  observations.  While  this  information  could  usually 
not  be  quantified,  it  provided  a  valuable  means  of  cross-checking  the  findings 
of  the  survey  and  provided  additional  illustrations  and  insights. 

During  the  research  numerous  examples  of  good  and  bad  practice  were 
encountered  or  described.  These  have  been  incorporated  into  the  study's 
recommendations. 
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EXECUTIVE  SUMMARY 


EXECUTIVE  SUMMARY 


Performance  measurement  and  capacity  planning  are  made  up  of  interrelated 
components,  as  shown  in  Exhibit  II- 1.  Two  of  these  components  stand  out  as 
particularly  critical  to  the  performance  measurement  and  capacity  planning 
process: 

Reporting  actual  versus  planned  performance  is  critical  for  effective 
capacity  planning. 

Capacity  planning  must  be  based  upon  the  business-motivated  require- 
ments of  computer  users. 

Performance  measurement  and  capacity  planning  have  become  increasingly 
important  in  the  last  several  years.  There  will  be  further  pressure  to  improve 
as  a  result  of  the  factors  shown  in  Exhibit  11-2. 

Computer  systems  are  becoming  more  important  to  many  firms. 

Accessibility  (i.e.,  response  and  turnaround  times)  is  increasingly 
critical. 

Many  of  the  factors  which  increase  the  need  for  planning  also  increase 
its  complexity. 
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EXHIBIT  ll-l 


COMPONENTS  OF  PERFORMANCE 
MEASUREMENT  AND  CAPACITY  PLANNING 
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EXHIBIT  11-2 
FORCES  DRIVING  CAPACITY  PLANNING 
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The  vast  majority  of  organizations  are  currently  struggling  to  establish  an 
effective  performance  measurement  program. 

Too  often  the  function  is  viewed  as  a  technical  process  divorced  from 
the  rest  of  the  MIS  organization,  as  well  as  from  users. 

One-third  of  installations  surveyed  do  not  yet  have  regularly  assigned 
staff  for  the  function. 

While  a  large  number  of  performance  measurement  tools  are  being 
used,  there  is  little  consistent  pattern  to  their  use;  many  installations 
admitted  that  they  are  not  receiving  full  benefits  even  from  their 
present  tools. 

While  benefits  were  cited  from  the  use  of  existing  tools,  in  many 
cases  they  were  not  based  on  reliable,  comprehensive  data. 

Similar  problems  exist  in  making  workload  projections. 

Installations  typically  cited  a  successful  record  in  making  projections. 
However,  the  baseline  data  to  make  solid  quantification  often  appeared 
to  be  lacking. 

Many  firms  have  a  "batch"  view  of  capacity  planning  rather  than  an 
"on-line"  view. 

They  see  their  computer  system  as  being  at  "X%"  of  capacity, 
rather  than  in  terms  of  response  or  turnaround  times. 

Many  performance  measurement  software  tools  are  now  offered,  as  well  as  a 
growing,  but  still  small,  number  of  capacity  planning  tools. 
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Unfortunately,  the  tools  are  usually  used  in  an  uncoordinated  and 
unintegrated  fashion,  often  complicating  the  problems  of  managing  a 
data  processing  installation. 

The  knowledge  and  use  of  capacity  planning  tools  are  quite  limited. 

It  is  doubtful  that  sophisticated  capacity  planning  can  be  done 
without  using  specialized  tools. 

To  be  effective,  each  function  of  performance  measurement  and  capacity 
planning  will  have  to  be  integrated,  as  shown  in  Exhibit  11-3. 

Software  tools  will  have  to  be  integrated. 

Capacity  planning  must  link  up  with  MIS  planning  in  general,  as  well  as 
with  overall  business  planning. 

Both  the  operations  and  development  sections  within  MIS  will  have  to 
work  with  users. 

However,  this  integration  cannot  stop  with  the  individual  functions,  but  must 
become  an  integrated  system,  as  shown  in  Exhibit  11-4: 

People. 

Planning. 

Tools. 

Performance  measurement  and  capacity  planning  can  be  powerful  facilitators 
of  a  system-wide  approach  to  data  processing. 
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EXHIBIT  11-3 
CAPACITY  PLANNING  FUNCTIONS 


TOOLS  PLANNING 
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THE  COMPONENTS  OF  AN 
INTEGRATED  CAPACITY  PLANNING  SYSTEM 
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Sufficient  capacity  is  what  both  MIS  management  and  user  management 
need.  Working  together  to  ensure  this  can  serve  as  a  means  of  having 
them  see  their  common,  rather  than  antagonistic,  interests. 


•         INPUT  recommends  that  MIS  management  attack  the  performance  manage- 
ment and  capacity  planning  problem  on  several  different  levels: 

Strategically. 

Gauge  the  level  of  effort  that  will  be  expended.  Too  much 
effort  may  sometimes  be  as  inappropriate  as  too  little;  however, 
current    experience    is    generally    inadequate,    rather  than 

excessive. 

Lay  the  foundation  for  later  progress  by  training  and  educating 
MIS  staff  and  users. 

MIS  management  should  take  an  active  role  in  capacity  planning. 
(It  will  be  held  responsible  for  failures  in  any  event.) 

Organizationally. 

Treat  capacity  planning  as  a  process  requiring  a  wide  range  of 
skills. 

Meld  people  of  different  backgrounds  by  using  a  task  force. 
Tactically. 

Set  up  plans  for  shedding  workloads. 

Consider  keeping  a  "tactical  reserve"  of  unused  resources  that 
can  be  called  on  in  times  of  need. 

Use  good  software  tools  (see  Appendix  D). 
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Ill   THE   CONCEPTUAL  FRAMEWORK 


THE  CONCEPTUAL  FRAMEWORK 


THE  IMPORTANCE  OF  PERFORMANCE  MEASUREMENT  AND 
CAPACITY  PLANNING 

Are  performance  measurement  and  capacity  planning  important?  Is  perform- 
ance measurement  only  a  justification  for  systems  programmers?  Is  capacity 
planning  only  an  exercise  that  is  gone  through  every  time  data  processing 
management  "knows"  that  it  needs  more  hardware?  Some  installations 
probably  still  operate  that  way,  but  many  other  MIS  managers  are  realizing 
that  measurement  and  planning  are  vital  for  carrying  out  their  missions. 

A  large  part  of  the  reason  for  the  increasing  importance  of  performance 
measurement  and  capacity  planning  is  that  data  processing  has  become  more 
successful  and  accepted,  so  much  so  that  it  has  become  critical  to  the 
profitability  of  many  organizations. 

Several  respondents  used  the  phrase  that  nowadays  it  is  a  "bet  your 
company"  situation  that  data  processing  would  perform  its  mission  as 
intended. 

One  company  estimates  that  $25,000  per  minute  of  cash  flow  is 
threatened  when  its  computer  system  is  down  during  peak  hours. 
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Another  company,  which  has  seen  its  3380  delivery  dates  slip  six 
months,  is  carrying  $3  million  per  day  of  interest  charges  on  a 
very  large  project  that  is  computer-dependent  in  many  of  its 
phases. 

For  many  organizations,  this  critical  position  has  been  reached  only 
recently  and  has  occurred  gradually  over  a  period  of  time.  Many  non- 
DP  managers  in  companies  where  data  processing  has  become  critical 
to  the  organization's  success  have  not  really  grasped  the  full  implica- 
tions of  the  changed  environment. 

Part  of  the  reason  for  this  delayed  acceptance  has  been  that  the  MIS 
department  itself  has  often  not  adjusted  to  the  changed  environment. 

Computer  operations  should  be  receiving  attention  and  resources 
at  least  on  a  par  with  the  development  unit.  However,  this  has 
not  happened.  Partly,  it  is  a  question  of  glamor. 

However,  there  is  also  a  perceptual  lag.  In  the  era  of  batch 
systems,  the  primacy  of  development  could  be  justified  on  the 
basis  that  system  development  was  very  difficult  and  also  was 
breaking  new  ground.  Operational  problems  could  usually  be 
resolved  with  the  application  of  brute  force. 

On-line  systems  present  an  entirely  different  set  of  issues.  Here 
successful  development  is  merely  the  first  step.  Afterward,  the  prime 
question  becomes  system  accessibility.  Namely: 

Is  the  system  available  when  needed? 

Is  response  time  adequate? 
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These  changes  are  shown  in  Exhibit  lll-l,  where  processing  time  has 
generally  shortened,  while  the  importance  of  accessibility  has 
increased. 

Note  that  special  queries  to  data  bases  will  often  have  a  high 
turnaround  time  priority  but  will  not  be  very  predictable.  Such 
queries  may  also  place  heavy  burdens  on  computer  resources. 

Most  users  now  "grade"  the  MIS  function  on  the  basis  of  accessibility. 
Since  larger  and  larger  sections  of  most  organizations  are  using  on-line 
systems,  MIS  departments  will  rise  or  fall  on  accessibility. 

Hence,  the  computer  operations  area  should  receive  an  equitable 
share  of  MIS  resources?  quantitative,  qualitative,  and 
intellectual. 

This  has  not  yet  widely  occurred,  as  is  apparent  in  visits  to  many 
data  processing  installations.  Additional  confirmation  is  shown 
by  the  generally  low  level  of  disaster  recovery  preparations  and, 
as  shown  in  this  study,  performance  measurement  and  capacity 
planning. 

•        Performance  measurement  and  capacity  planning  are  among  the  chief  methods 
of  ensuring  proper  accessibility  of  computer  resources. 


B.       COMPONENTS  OF  PERFORMANCE  MEASUREMENT  AND  CAPACITY 
PLANNING 


•  The  terms  "performance  measurement"  and  "capacity  planning"  are  shorthand 
for  an  interrelated  process  which  starts  with  performance  measurement  and 
ends  with  capacity  planning.  Exhibit  111-2  shows  this  process  in  pictorial  form, 
with  performance  measurement  as  the  foundation  and  capacity  planning  the 
capstone. 
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EXHIBIT  111-1 


CHANGING  PATTERNS  IN  COMPUTER  PROCESSING 
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-  REPORTS  TO  USERS 

•  DATA  BASE  INTERROGATION 

-  REGULAR  REPORTING 

-  SPECIAL  "WHAT  IF'1  QUERIES, 

ETC. 

•  INTERACTIVE  TRANSACTION 

PROCESSING 

-  EDIT  TURNAROUND 

-  DATA  BASE  UPDATE 

-  REPORTS  TO  USERS 


MEDIUM 

HIGH 

HIGH 


HIGH 
HIGH 
HIGH 


HIGH 


MEDIUM/LOW 


HIGH 
HIGH 
HIGH 


DAYS 

WEEKS 

WEEKS 


HOURS 

DAYS 

DAYS 


HOURS 


HOURS/DAYS 


SECONDS 
SECONDS 
HOURS 
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EXHIBIT  1 1 1  —  2 


BUILDING  CAPACITY  PLANNING  CAPABILITY 


I  

PROJECTING 

CAPACITY 
REQUIREMENTS 


TRANSLATING  BUSINESS 

NEEDS  INTO 
COMPUTER  WORKLOADS 


TUNING  AND 
SCHEDULING  THE  SYSTEM 


COMPARING  ACHIEVEMENTS 
TO  STANDARDS 


SETTING  PERFORMANCE  STANDARDS 


Measuring  performance 
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This  entire  process  is  sometimes  given  other  names,  such  as  perform- 
ance management,  to  connote  the  integrative  aspects  of  the  process. 

Each  of  the  levels  shown  in  Exhibit  111-2  builds  on  the  previous  ones.  This 
section  will  provide  a  brief  overview  of  each  level.  The  remaining  sections  of 
this  chapter  will  deal  with  each  level  in  detail. 

Measuring  performance  provides  the  baseline  data  required  for  setting 
standards  or  making  improvements.  Otherwise,  "You  can't  get  there 
from  here."  There  are  numerous  software-  and  hardware-based 
measuring  tools  to  assist  in  this  process. 

Once  a  baseline  describing  current  performance  has  been  defined,  it  is 
possible  to  negotiate  the  setting  of  performance  standards  with  users. 
This  process  is  in  its  infancy  in  most  organizations.  However,  it  is 
required  in  order  to  know  how  well  the  system  is  performing. 

Comparing  achievements  to  standards  becomes  relatively  straight- 
forward once  the  earlier  steps  have  been  completed.  At  this  point,  it  is 
possible  to  begin  making  capacity  need  projections  by  establishing  the 
extent  to  which  performance  standards  have  been  met. 

Tuning  and  scheduling  make  sense  only  if  they  can  be  compared  to 
unambiguous  baseline  performance  data  and  to  a  performance  standard. 
All  too  often,  current  tuning  and  scheduling  take  place  in  a  vacuum 
with  no  real  knowledge  of  how  current  achievements  compare  to  the 
historic  record  or,  more  importantly,  how  well  (or  how  poorly)  user 
needs  are  being  satisfied. 

The  most  difficult  step  in  preparing  a  capacity  plan  is  translating 
business  needs  into  computer  workloads.  There  are  several  aspects  to 
this: 
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Trends,  especially  seasonal  variations,  as  shown  in  historic  data. 

Business  plans  which  would  modify  the  historic  data  (either  in 
the  rate  of  growth  or  seasonality).  For  example,  an  application 
may  receive  significant  enhancements  or  additional  users  and 
require  substantially  more  computer  resources. 

New  applications  being  developed.  Some  are  similar  to  existing 
applications  and  may  consequently  be  planned  by  analogy. 
Others,  however,  may  be  totally  new  and  previous  experience 
would  not  be  useful. 

Finally,  there  may  be  underlying  changes  in  the  business  that 
would  have  profound  effects  on  computer  requirements  (acquisi- 
tions, new  lines  of  business,  geographic  expansion  or  contraction, 
etc.). 

As  can  be  seen,  at  a  certain  point  effective  capacity  planning 
must  become  a  subset  of  overall  business  planning  and  informa- 
tion system  planning. 

Finally,  if  all  the  data  and  analyses  of  prior  steps  have  been  carved  out 
adequately,  it  will  be  possible  to  project  capacity  requirements  of  the 
hardware  configuration. 

There  are  several  recently  offered  software  aids  which  promise 
to  make  the  mechanics  of  this  step  faster  and  more  straight- 
forward. However,  the  value  of  capacity  planning  software  is 
largely  dependent  on  a  solid  factual  base  informed  by  intelligent 
analysis. 

Hardware  issues  will  soon  become  more  complex  as  distributed 
processing  networks  and  standalone  minicomputers  become  more 
significant  factors. 
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•  The  succeeding  sections  of  this  chapter  will  discuss  each  component  in  more 
detail  and  provide  much  of  the  conceptual  framework  for  later  discussion  of 
the  current  status  in  the  field,  future  trends,  and  INPUT'S  recommendations  to 
data  processing  installations. 

This  conceptual  framework  is  not  theoretical  but  is  based  upon  situa- 
tions observed  while  conducting  this  study,  as  well  as  the  observations 
and  analyses  of  those  who  have  spent  many  years  working  in  or 
reporting  on  this  field. 


C.       PERFORMANCE  MEASUREMENT 


•  There  are  a  large  number  of  aids  and  tools  to  assist  in  collecting  and 
presenting  data  on  system  performance.  Tools  fall  into  the  following  general 
categories  (although  there  is  sometimes  overlap  between  categories  for 
specific  tools): 

Job  accounting  packages. 

Performance  monitors. 

General-purpose  software  monitors. 

Specialized  software  monitors. 

Hardware  monitors. 

Capacity  planning  software  (to  be  discussed  more  fully  in  Section  H  of 
this  chapter). 

•  Exhibit  1 1 1-3  provides  an  overview  of  the  different  types  of  tools,  their 
function,  and  the  major  uses  to  which  they  are  put. 
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EXHIBIT  1 1 1  —  3 
TOOL  TYPES  AND  USES 


TOOL  TYPE 

FUNCTION 

EXAMPLES  OF 
MAJOR  USES 

JOB  ACCOUNT- 
ING PACKAGE 

•  DESCRIBES  USE  OF  SYSTEM, 

BY  TYPE  OF  USE  AND 
USER 

•  DATA  SOURCE:  SMF  DATA 

TYPICALLY 

•  CHARGEBACK 

•  BUDGETING 

•  FIRST-LEVEL  PERFORM- 

ANCE MONITORING 

PERFORMANCE 
MONITOR 
(IN  GENERAL) 

•    REPORTS  ON  SYSTEM 
PERFORMANCE 

•  PERMITS  MORE  COMPREHEN- 

SIVE PERFORMANCE 
MONITORING 

•  PROVIDES  BASELINE  DATA 

rUK            1  tm  lUlNIINO 

•  PROVIDES  HISTORIC  DATA 

FOR  FIRST-LEVEL 
CAPACITY  PLANNING 

GENERAL-PUR- 
POSE SOFT- 
WARE 
MONITOR 

•  DESCRIBES  PERFORMANCE 

OF  OVERALL  SYSTEM 

•  DATA  SOURCE:  RMF  DATA 

TYPICALLY 

•    SYSTEM  TUNING 

SPECIALIZED 
SOFTWARE 
MONITOR 

•  DESCRIBES  PERFORMANCE 

OF  A  SYSTEM  COMPONENT 

•  DATA  SOURCE  VARIES 

•    MONITORING  AND  TUNING 
OF  PARTICULAR  SUBSYS- 
TEMS; E.G.,  DASD  SPACE, 
IMS,  TSO,  ETC. 

HARDWARE 
MONITOR 

•  DESCRIBES  PERFORMANCE 

OF  COMPLEX  SYSTEMS 

•  DATA  SOURCE:  SPECIAL 

PHYSICAL  PROBES 

•  MONITORING  AND  TUNING 

LOOSELY  COUPLED  CPUs 

•  COMMUNICATIONS  NET- 

WORK MONITORING 
AND  OPTIMIZATION 

CAPACITY 
PLANNING 
SOFTWARE 

•  DESCRIBES  EFFECTS  OF 

WORKLOAD  CHANGE  ON 
HARDWARE  REQUIREMENTS 

•  DATA  SOURCES:    (A)  SMF/ 

RMF/MON ITORS  (B)  ESTI- 
MATES OF  FUTURE  LOADS 

•  PROJECT  EFFECTS  OF 

A  NEW  APPLICATION 
SYSTEM 

•  PROJECT  EFFECTS  OF 
ADDITIONAL  USE  OF 
A  CURRENT  SYSTEM 
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•  One  of  the  questions  which  arises  in  performance  measurement  is  the 
differences  between,  and  advantages  of,  hardware  versus  software  monitors. 
Exhibit  111-4  summarizes  the  major  differences. 

In  general,  software  monitors  are  easier  to  use  and  do  not  require  highly 
skilled  personnel. 

Software  monitor  acquisition  cost  is  much  less  expensive  (sometimes  by 
an  order  of  magnitude).  Formerly,  software  monitors  imposed  a 
significant  overhead  cost  on  the  system  (upwards  of  10%).  However, 
newer  software  monitors  may  impose  little  more  than  1%.  (The  actual 
overhead  will  vary,  depending  on  how  often  they  are  used  and  how  much 
detail  is  collected.) 

Hardware  monitors  are  by  their  nature  more  precise  than  software 
monitors,  since  they  are  collecting  electrical  impulses  rather  than 
sampling  sometimes  ambiguous  system  or  program  events. 

However,  making  sense  of  the  data  collected  by  hardware 
monitors  sometimes  requires  a  high  degree  of  experience,  i.e.,  it 
is  as  much  of  an  art  as  a  science. 

However,  there  are  certain  situations  which  only  a  hardware  monitor 
can  measure,  most  notably,  loosely  coupled  CPUs  with  shared  direct 
access  storage  devices  (DASD). 

This  is  due  to  the  hardware  monitor's  ability  to  have  its  probes 
connected  to  a  number  of  semi-independent  points  in  the  config- 
uration and  to  integrate  the  data  received. 

Software  monitors  residing  in  different  CPUs,  on  the  other  hand, 
cannot  feasibly  have  their  outputs  meshed;  a  software  monitor 
resident  in  one  CPU  cannot  measure  what  is  occurring  in  another 
CPU. 
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EXHIBIT  1 1 1  —  a 


DISTINCTIONS  BETWEEN  HARDWARE 
AND  SOFTWARE  MONITORS 


MONITOR  TYPE 

CHARACTERISTICS 

HARDWARE 

SOFTWARE 

•    MEASUREMENT  TYPE 

CONTINUOUS* 

SAMPLING 

•    PROCESSOR  INDEPENDENCE 

-    SIMULTANEOUSLY  MEASURES 
MULTIPLE  CPUs  WITH  SHARED 
DASD? 

YES 

NO 

-    MEASUREMENTS  AFFECTED  BY 
PROCESSOR  FAILURE? 

NO 

YES 

-    MONITOR  IMPOSES  OVERHEAD 
ON  PROCESSOR? 

NO 

YES 

(2-5%) 

-    MONITOR  CAPTURES  APPLICATION- 
SPECIFIC  DATA  (E.G.,  PROGRAM 
ID)  ? 

NO 

YES 

-    MONITOR  PORTABLE  BETWEEN 

DIFFERENT  OPERATING  SYSTEMS 
AND  VENDORS'  HARDWARE? 

YES 

NO 

(USUALLY) 

•    EASE  OF  USE 

-    REQUIRES  KNOWLEDGE  OF 

PROCESSOR  ARCHITECTURE? 

YES 

NO 

(USUALLY) 

-    REQUIRES  SUBSTANTIAL  INITIAL 
TRAINING? 

YES 

NO 

-    REQUIRES  HIGHLY  SKILLED 
PERSONNEL? 

YES 

NO 

-    MONITOR  HIGHLY  FLEXIBLE? 

YES 

(IN  PRINCIPLE) 

NO 

•  PRICE 

HIGH 
($60,000-400,000) 

LOW /MODERATE 
($5,000-25,000) 

"UNDERLINING  DENOTES  THE  MORE  VALUABLE  CHARACTERISTIC 
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However,  hardware  monitors  are  not  the  complete  answer.  They  only 
measure  externally  gathered  electrical  impulses  and,  not  being  resident 
in  the  CPU,  cannot  associate  these  measurements  with  particular 
application  systems. 

The  ideal  (although  it  might  be  very  expensive)  would  be  an  integrated 
hardware/software  monitor;  this  does  not  now  exist. 

Even  if  such  an  improved  monitor  did  exist  it  would  be  useful  only  in 
installations  that  had  succeeded  in  wringing  most  of  the  useful  informa- 
tion out  of  present  monitors. 

A  similar  distinction  should  be  made  between  job  accounting  packages  and 
software  monitors. 

Job  accounting  packages  perform  well  for  what  they  are  intended  to  do: 
describe  the  activity  associated  with  individual  jobs  or  subsets  and  supersets  of 
jobs. 

However,  for  the  most  part,  job  accounting  packages  are  based  on  SMF- 
type  data  (or  whatever  the  non-IBM  hardware  manufacturer  may  call  it) 
which  looks  at  individual  job  or  job  stop  performance. 

Software  monitors,  on  the  other  hand,  access  RMF  data,  which  looks  at 
the  system  as  a  whole.  Thus,  the  typical  general-purpose  software 
monitor  can  give  a  much  deeper  and  broader  view  of  the  system  than  a 
job  accounting  package  alone  can.  Exhibit  1 11-5  shows  these  contrasts 
for  typical  capabilities. 

Note,  for  example,  that  for  the  key  metric  of  response  time  a  software 
monitor  is  needed. 

The  genera!  principles  described  here  exist  for  both  the  IBM  and  non-IBM 
world. 
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EXHIBIT  i  1 1  —  5 


CONTRASTS  IN  CAPABILITIES  OF  JOB 
ACCOUNTING  PACKAGES  AND  SOFTWARE  MONITORS 


REPORTED  BY 

REPORTING 

CAPABILITY 

JOB 
ACCOUNTING 
PACKAGE 

SOFTWARE 
MONITORS 

TURNAROUND  ELAPSED  TIME 

X 

X 

RESPONSE  TIME 

X 

CPU  UTILIZATON 

X 

X 

OVERLAPS,  WAITS 

X 

EXCP  ACTIVITY 

X 

X 

CHANNEL  UTILIZATION 

X 

PAGING  ACTIVITY 

X 

X 

WORKING  SET  SIZE 

X 

DISK  EXCPs 

X 

DISK  QUEUES,  FREE  SPACE 
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All  hardware  manufacturers  offer  a  built-in  collection  system  analogous 
to  IBM  SMF/RMF  capabilities.  However,  beyond  manufacturer-supplied 
collection  and  reporting  facilities  there  is  not  a  great  deal  available  to 
non-IBM  installations: 

In  the  course  of  preparing  this  report,  INPUT  analyzed  a  large 
subset  of  software  products  currently  available  for  performance 
measurement  and  improvement.  Not  surprisingly,  over  three- 
quarters  of  these  were  aimed  at  IBM  users,  as  shown  in  Exhibit 
1 1 1-6. 

Other  manufacturers  are  concerned  about  this  problem  and  realize  that 
they  must  provide  adequate  support  for  their  customers'  performance 
measurement  and  capacity  planning: 

Burroughs,  for  example,  introduced  in  April  1980  a  significant 
enhancement  to  its  SPARK  package. 

Besides  improving  existing  reporting  on  workload  character- 
ization and  utilization  measurement,  SPARK  will  report  on 
detected  hardware  faults  and  overall  system  availability. 

Tandem  has  built  much  of  the  performance  data  collection 
facility  into  its  computer  systems.  This  monitor  (XRAY)  is 
unwieldy  and  needs  a  sophisticated  systems  programmer-type 
user.  However,  some  Tandem  users  don't  have  any  programmers. 

Obviously,  minicomputer  companies  like  Tandem  are  going  to 
have  to  take  a  different  approach  from  the  mainframe  manufac- 
turers. 

However,  except  for  the  range  of  products  offered  to  IBM  users,  the 
issues  and  solutions  do  not  differ  markedly  between  mainframe  man- 
ufacturers.   The  resources  and  sophistication  available  to  an  installa- 
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EXHIBIT  1 1 1  —  6 


APPLICABILITY  OF  PERFORMANCE  MEASUREMENT 
AND  IMPROVEMENT  SOFTWARE  PRODUCTS 
TO  PARTICULAR  HARDWARE  ENVIRONMENTS 


TOTAL  PRODUCTS  AVAILABLE  =  159 

SOURCE:  INPUT  TABULATION 
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tion,  and  the  intelligence  applied  to  problems,  are  typically  more 
important  factors  than  the  type  of  hardware  employed. 

D.       PERFORMANCE  STANDARDS 

•  Performance  standards  must  be  established  to  reflect  business  reality;  this 
means  serious  negotiations  with  the  users  involved. 

For  most  users,  accessibility  is  the  key  issue: 

Response  time  for  interactive  system. 

Turnaround  time  for  batch  and  RJE  systems. 

However,  there  is  a  real  price  associated  with  most  improvements  and 
thorough  analysis  must  be  undertaken  to  see  if  the  economic  benefits 
balance  the  increased  costs. 

•  A  semiautomatic  method  to  effect  the  negotiations  and  external  analysis 
required  in  setting  performance  standards  would  be  very  appealing. 

•  One  approach,  which  has  considerable  intellectual  appeal,  is  to  allocate 
computer  resources  by  a  pricing  mechanism,  allowing  users  to  decide  for 
themselves  whether  a  certain  level  of  service  is  economically  justified. 

However,  there  are  practical  problems,  illustrated  in  the  situation 
shown  in  Exhibit  111-7.  This  shows  an  on-line  system  with  its  classic 
twin  usage  peaks.  Theoretically,  a  time-of-day  chargeback  system 
(based  on  data  collected  by  the  job  accounting  system)  would  spread 
usage  throughout  more  of  the  day.  However  in  some  cases  a  charge- 
back system's  success  has  led  to  its  ultimate  failure: 
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EXHIBIT  111-7 

PROPORTION  OF  A 
COMPUTER  SYSTEM  CONSUMED 
BY  AN  ON-LINE  APPLICATION 


6  8         1  0        12         2  4         6  8        10  12 

A.M.  P.M. 


TIME  OF  DAY 
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If  the  underlying  application  is  an  order  entry  system,  for 
example,  it  would  be  counterproductive  to  discourage  order 
clerks  from  using  the  system  immediately  to  look  up  stock 
status,  quote  additional  prices,  enter  cancellations,  etc. 

System  costs  might  be  lower  if  these  "marginal"  functions  were  postponed  to 
an  off-peak  time;  however,  the  usefulness  of  the  system  -  and  its  impact  on 
the  bottom  line  -  would  be  negatively  affected. 

In  many  other  cases  a  chargeback  system  would  have  only  minimal 
effect.  Take,  for  example,  an  interactive  user  system  or  a  TSO 
program  development  system: 

Within  very  broad  limits,  users  are  insensitive  to  the  cost  of 
computer  time  compared  to  personnel  expense,  deadlines,  etc. 
(the  charges  are  usually  a  form  of  "funny  money"  anyway). 

If  it  is  really  important  to  off-load  such  work,  then  pricing 
becomes  a  rather  minor  weapon  in  an  armory  which  would 
include  such  things  as  flextime,  terminals  at  home,  exhortation 
(or  threats),  and  good  off-hour  response  time  (since  all  too  often 
response  at  7  p.m.  may  be  no  better  than  at  2  p.m.  because  of 
large  batch  jobs  or  maintenance). 

The  job  of  negotiating  with  users  becomes  one  of  rationing  a  scarce,  but 
"free,"  resource. 

Would  it  be  better  to  avoid  battles  and  not  establish  quantified  and 
auditable  performance  standards? 

Yes,  perhaps  in  the  short  run,  especially  if  the  organization  as  a 
whole  does  not  or  cannot  plan  effectively. 
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However,  in  the  longer  run  there  is  no  avoiding  the  issue,  since 
the  user  will  always  have  implicit  service  standards  that  MIS  will 
be  judged  by.  In  some  cases  a  frank  discussion  of  the  issues 
involved  will  result  in  explicit  standards  that  are  easier  to  meet. 

In  a  "rationing"  atmosphere  it  becomes  critical  to  involve  top  manage- 
ment. One  of  management's  roles  is  to  allocate  resources  or,  in  this 
case,  to  approve  an  approach  for  allocating  resources. 

E.       COMPARING  ACHIEVEMENTS  TO  STANDARDS 

•  Contrasting  actual  performance  against  previously  agreed  upon  standards  is  a 
job  that  will  never  end.  The  data  processing  department  does  not  want  to 
either  exceed  or  fall  short  of  the  standard  significantly. 

To  routinely  exceed  the  standard  may  give  rise  to  a  new  de  facto 
standard,  which  may  not  be  needed. 

•  Regular  reports  of  actual  versus  planned  performance  should  be  provided  to  all 
user  departments.  Great  care  should  be  taken  to  ensure  that  the  reports  are 
meaningful  and  that  both  sides  agree  on  the  definition  of  a  particular  standard. 
Definition  and  selectivity  are  very  important  in  performance  reporting,  but  it 
can  sometimes  be  difficult  to  arrive  at  meaningful  reports.  Take,  for 
example,  the  key  metric  of  system  availability: 

It  sometimes  seems  that  users  and  MIS  inhabit  different  worlds  (or  at 
least  use  different  computers). 

Users  may  complain  about  their  system  "always"  being  down. 

MIS  management  reports  that  the  system  is  up,  say,  98%  or  99% 
of  the  time. 
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They  may  both  be  right,  according  to  their  separate  definitions. 

The  user's  figure  is,  of  course,  very  impressionistic.  However,  upon 
analysis,  suppose  a  typical  terminal  is  shown  to  be  unavailable  for  use 
during  20%  of  the  prime  shift. 

Data  processing  operations,  on  the  other  hand,  measure  the  time  the 
CPU  is  available  (less  eight  hours  scheduled  maintenance);  its  logs  show 
a  figure  of  98.7%. 

However,  the  CPU  is  of  little  use  to  a  user  unless  the  entire  hardware/ 
software  configuration  is  available.  Exhibit  III — 8  lists  the  typical 
components  of  a  system  which  an  on-line  application  would  use  and 
shows  illustrative  reliability  factors. 

Each  component  is  assumed  to  have  a  high  degree  of  reliability. 

However,  the  system's  reliability  is  only  87%. 

If  the  reasonable  assumption  is  made  that  a  disproportionate  number  of 
failures  occur  during  the  prime  shift,  then  the  reported  1%  and  20% 
failure  rates  are  in  fact  describing  the  same  universe. 

This  kind  of  measurement  is  often  not  performed  because  it  is  much  harder  to 
measure  total  system  availability  and  even  harder  to  measure  availability  of 
individual  components.  In  addition,  the  psychic  rewards  to  MIS  management 
are  much  less.  MIS  management  would  have  to  adjust  to  performing  at  an 
order  of  magnitude  worse  than  they  had  previously  expected. 

However,  without  such  realism,  it  will  be  impossible  for  MIS 
management  to  have  meaningful  dialogues  with  users  and  to  take 
constructive  action  on  real  problems. 
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EXHIBIT  IM-8 


COMPUTER  SYSTEM  VERSUS  COMPONENT  RELIABILITY 

(HYPOTHETICAL  EXAMPLE) 


COMPONENT 

UP-TJME 
(PERCENT)* 

HARDWARE 

INDIVIDUAL  TERMINAL 

99% 

MODEMS 

99 

LINES 

99 

MULTIPLEXOR 

99 

CONTROLLER 

99 

DASD 

99 

CHANNEL 

99 

CPU 

99 

TOTAL  HARDWARE 

92% 

SOFTWARE 

DBMS 

98% 

TP  CONTROL  PROGRAM 

99 

OPERATING  SYSTEM 

99 

APPLICATION  SOFTWARE 

99 

TOTAL  SOFTWARE 

95% 

TOTAL  PROCESSING  SYSTEM 

87% 

*  FIGURES  FOR  ILLUSTRATION  ONLY. 


-31  - 

©1981  by  INPUT.  Reproduction  Prohibited. 


•        Because  of  the  clanger  of  regular  reporting  going  unintentionally  askew,  other 
means  of  feedback  from  users  should  be  formalized  and  encouraged. 

A  quarterly  "report  card"  from  users,  for  example,  will  often  bring  out 
unsuspected  problems,  as  well  as  open  another  means  of 
communication. 


F.       TUNING  AND  SCHEDULING 


•  Most  installations  are  already  doing  a  considerable  amount  of  tuning  and 
scheduling. 

However,  even  most  advanced  installations  are  just  beginning  to  set 
meaningful  performance  standards.  This  raises  the  question  of  whether 
there  are  specific  objectives  for  present  tuning  and  scheduling. 

•  Much   existing   tuning   either   occurs   inside   a   closed   system   or  entails 
"firef  ighting." 


"Closed  system"  tuning  is  where  a  systems  programmer  analyzes 
performance  measurement  reports  and  sees  an  opportunity,  for 
example,  to  reorganize  DASD  data  sets  and  reduce  average  response 
time  by,  say,  one  second  for  a  particular  application. 

This  is  good  as  an  end  in  itself  since  everyone  in  data  processing 
(especially  systems  programmers)  likes  efficiency.  But  did  that 
application  need  the  improvement?  Perhaps  that  particular 
application  could  have  had  other  priorities  lowered  (so  it 
received  no  net  benefit),  with  the  freed  resources  used  to  help 
another  application. 
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"Firefighting"  is  self-explanatory.  The  question  is:  Need  it  have 
occurred?  Would  better  performance  measurement  and  analysis  have 
identified  the  problem  before  the  critical  stage?  Or  were  there  no 
performance  standards  against  which  to  make  the  judgment  that  the 
critical  stage  was  approaching? 

G.      TRANSLATING  BUSINESS  NEEDS  INTO  COMPUTER  WORKLOADS 

•  Translating  business  needs  into  overall  computer  solutions  should  be  one  of  the 
main  concerns  of  the  MIS  department.  If  this  is  so,  then  one  important  part  of 
the  "computer  solution"  is  estimating  hardware  requirements.  However,  this 
state  has  not  been  reached  in  a  great  many  computer  installations. 

•  User  departments  and/or  the  corporate  planning  function  should  be  the  best 
source  for  information  on  projections  for  such  things  as: 

Increased  business  by  area  and/or  product  line. 

New  business  plans. 

Changes  in  product  or  service  composition  that  would  affect  data 
processing  needs. 

Seasonal  business  patterns;  current  as  well  as  potential  changes. 

Present  and  projected  geographic  patterns  of  business. 

The  MIS  department  will  then  have  to  take  these  needs  (sometimes  called 
"natural  business  units,"  "forecast  units,"  etc.)  and  translate  them  into 
computer  resource  equivalents. 


-33  - 

©1981  by  INPUT.  Reproduction  Prohibited. 


INPUT 


Software  tools,  described  in  the  following  section  of  this  chapter,  can 
be  of  great  assistance. 

However,  the  basic  problem  is  one  of  understanding,  and  should  not  be 
underestimated.  A  very  rare  and  high-quality  person  is  needed  for  this 
kind  of  work. 

At  this  point,  some  readers  will  be  thinking,  "We  have  high-powered  systems 
analysts  and  architects  who  really  understand  the  business  and  are  already 
performing  just  this  kind  of  translation." 

The  problem  may  be  that  they  are  performing  a  different  sort  of 
translation.  They  are  looking  at  logical  processes,  data  base  designs, 
and,  hopefully,  user-friendly  systems. 

They  often  do  not  care  about  operations  requirements  ("Buy 
another  machine"). 

Exhibit  111-9  is  an  attempt  to  quantify  the  priorities  for  most  system 
development  personnel. 

They  may,  in  fact,  be  quite  correct  in  relegating  computer 
resource  questions  to  the  backs  of  their  minds.  A  million  dollars 
or  more  of  additional  hardware  may  be  a  perfectly  acceptable 
trade-off  for  a  user-accessible  data  base,  maintainable 
programs,  etc. 

However,  data  processing  operations  may  not  have  an  inkling  of  the 
magnitude  of  resources  required  until  it  is  far  too  late  to  do  such  things 
as: 

Order  required  hardware. 
Optimize  code. 
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EXHIBIT  111-9 


HYPOTHETICAL  PRIORITIES  OF  A 
SYSTEM  DEVELOPMENT  GROUP 


CRITERIA 

IMPORTANCE 

FINISH  ON  SCHEDULE 

10 

FINISH  WITHIN  BUDGET 

9 

MEET  USERS'  FUNCTIONAL  REQUIREMENTS 

8 

MAXIMIZE  STAFF  PRODUCTIVITY 
(E.G.,  LINES  OF  CODE  PER  DAY) 

8 

PRODUCE  MAINTAINABLE  SYSTEMS 

6 

DEVELOP  AN  INTEGRATED  DATA  BASE 

6 

MEET  RESPONSE  TIME  REQUIREMENTS 

6 

MINIMIZE  COMPUTER  RESOURCES  NEEDED 

2 

(10  =  HIGH  PRIORITY,  1  =  LOW  PRIORITY) 
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Off-load  other  systems,  etc. 


The  bottom  line  is  that  the  development  group  usually  cannot  be 
expected  to  look  after  operations'  interests. 

H.       PROJECTING  CAPACITY  REQUIREMENTS 


•  Some  installations  apparently  still  project  their  requirements  by  assuming  that 
the  future  consists  of  a  straight  line  (or  at  the  worst,  a  slightly  curved  line). 
Any  installation  which  uses  only  regression  analysis  for  capacity  planning  is  in 
this  category.  Exhibit  111-10  shows  how  people  with  this  point  of  view  would 
implicitly  view  workload  projection  and  capacity  planning.  (They  probably 
would  also  use  CPU  capacity  resource  units  as  their  measure  of  capacity.) 

•  Real  life  is  far  more  complicated.  Exhibit  III- 1  I  shows  the  kinds  of  curves 
that  reflect  actual  conditions,  and  Exhibit  111-12  shows  a  few,  quite  simplified, 
possible  growth  scenarios. 

Exhibit  111-13  quantifies  the  scenarios'  assumptions. 

•  Two  sets  of  scenarios  are  charted  in  Exhibit  111-14. 

All  scenarios  (Group  "I")  growing  a  total  of  21%  over  18  months. 

The  higher  growth  scenarios  (B,  C,  D,  H,  or  Group  "II")  growing  76% 
over  1 8  months. 

Both  groups  show  episodic  growth  in  transactions  processed. 
Group  I  ranges  from  6%  to  12%  per  six-month  period. 
Group  II  ranges  from  9%  to  19%  per  six-month  period. 
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EXHIBIT  111-11 
CAPACITY  CURVES  (SCHEMATICS) 


NEW  APPLICATION  (OR  OLD  NEW  FUNCTIONS  INTRODUCED 

APPLICATION  WITH  GROWING  TO  OLD  SYSTEM 

NUMBER  OF  USERS) 


APPLICATION  PHASING  OUT  AS  ABOVE,  WITH  THE  NUMBER  OF 

TERMINALS  ALSO  INCREASED 


FUNCTIONS/USERS  OFF-LOADED  ONTO 
STANDALONE  MINIS 


NEW  DEPARTMENT  ADDED 

(A)     INITIAL  LEARNING  CURVE, 


HIGH  ENTHUSIASM 

(B)      MORE  EFFICIENT  USE,  "JOYRIDERS 
DROPOFF 
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EXHIBIT  111-12 


GROWTH  SCENARIOS 


SCENARIO 

DESCRIPTION  OF  COMPUTER  RESOURCE  USAGE 
PER  SIX-MONTH  INTERVAL 

A 

STEADY  3%  GROWTH  IN  TRANSACTIONS 

B 

STEADY  10%  GROWTH  IN  TRANSACTIONS 

C 

10%  GROWTH,  WITH  ENHANCEMENTS 

AT    CIV  —  MflMTU     IMTCDX/AI  C 
A  1    bl  A  IviUIN  In    I  IN  1  tKVALo 

D 

NO  TRANSACTION  GROWTH,  BUT  SIGNIFICANT 
ENHANCEMENTS  AT  SIX-MONTH  INTERVALS 

E 

SIGNIFICANT  OFF-LOADING  OF  TRANSACTIONS 
TO  STANDALONE  MINICOMPUTER  AT  SIX- 
MONTH  INTERVALS 

F 

AS  ABOVE,  BUT  A  10%  DECLINE  IN  TRANSACTION 
VOLUME  AFTER  EACH  OFF-LOADING 

G 

SIGNIFICANT  GROWTH  IN  TRANSACTIONS  DUE  TO 
NEW  DEPARTMENT  USING  TIMESHARING  SYSTEM; 
SYSTEM  ENHANCED  AFTER  SIX  MONTHS,  BUT 
DEPARTMENT  THEN  DECIDES  TO  USE  AN  IN- 
HOUSE  MINI  INSTEAD;  MODEST  GROWTH 
THEREAFTER. 

H 

NEW  APPLICATION;  400%  GROWTH  (FROM  A  SMALL 
BASE)  LEVELING  OFF  TO  50%  AND  THEN  25%. 
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EXHIBIT  111-14 


GROWTH  SCENARIOS 


'ASSUMES  2-MONTH  phase-in/out 
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In  addition,  there  are  sudden  increases  and  decreases  as  systems  are 
added  or  dropped. 

In  neither  case  would  regression  equations  be  of  much  assistance. 

If  seasonal  fluctuations  were  introduced  (conforming  to  the  underlying  trends 
shown)  which  had  different  cycles  and  amounts  of  fluctuation  for  different 
applications,  then  the  analysis  would  become  very  difficult,  perhaps 
impossible,  to  perform  manually. 

Until  recently,  the  number  and  kinds  of  software  packages  available  to  assist 
in  capacity  planning  were  fairly  limited.  The  best  known  was  IBM's  SNAP 
SHOT,  a  simulation  package  used  by  IBM  for  marketing  support. 

This  meant  that  customers  could  use  it  only  at  IBM  facilities  and  could 
not  buy  or  rent  it. 

Even  more  of  a  limitation  for  many  customers  is  the  detailed  amount  of 
data  that  must  be  collected,  the  special  trip  to  an  IBM  site  often 
required,  and  the  lengthy  run  times. 

IBM  responds,  quite  correctly,  that  the  result  is  a  very  precise  picture 
of  machine  requirements. 

There  are  now  software  alternatives  that  bear  serious  looking  into.  The 
modeling  package,  Best/I  (by  BGS  Systems,  Waltham,  MA),  for  example,  has 
found  considerable  acceptance  since  first  being  offered  in  1978. 

It  follows  a  modeling,  rather  than  simulation,  approach.  RMF/SMF  data 
are  combined  with  models  of  hardware  configurations  to  develop  a 
baseline  case,  as  shown  in  Exhibit  111-15.  This  means,  at  least 
theoretically,  that  there  is  a  trade-off  between  precision  and  ease  of 
use. 


-42  - 

©1981  by  INPUT.  Reproduction  Prohibited. 


INPl 


EXHIBIT  111-15 


THE  MODELING  APPROACH  TO  CAPACITY  PLANNING 
(AS  EXEMPLIFIED  IN  THE  BEST/I  PACKAGE) 


INPUTS 


WORKLOAD  HIS- 
TORY 

-  RMF  DATA 

-  SMF  DATA 

-  OTHER  LOG 
DATA  (E.G.# 
IMS) 

-  OTHER  MONI- 
TOR INPUT 


OUTPUTS: 


WORKLOAD 

VARIABLES 

-  JOB  ARRIVAL 
RATE 

-  AVERAGE/ 
MAXIMUM 
LEVELS  OF 
MULTIPRO- 
GRAMMING 


•  HARDWARE 

PERFORMANCE 
CHARACTERISTICS 

-  EXISTING  CON- 

FIGURATION 

-  OPTIONS 


(EXAMPLES) 

•  RESPONSE  AND  THROUGHPUT 

TIMES 

•  WAITING  TIME  AND  QUEUE  LENGTHS 

•  MEMORY  UTILIZATION 
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To  some  extent,  this  is  like  the  distinction  between  precision  and 
accuracy  in  scientific  experiments:  What  good  does  it  do  to  take 
measurements  to  a  precision  of  .0001  if  the  accuracy  of  the 
instruments  is  .01? 

The  analogy  in  capacity  planning  is  whether  simulation  that  may 
give  results  to,  say,  +2%,  is  preferable  if  underlying  key  business 
or  workload  estimates  vary  by  +20%. 

The  importance  of  these  theoretical  issues  will  vary  from 
installation  to  installation  depending  on  the  accuracy  of  the 
characterization  of  workload. 

The  attractions  to  its  users  of  a  product  like  Best/I  are  practical  and 
operational: 

It  can  be  purchased  and  used  completely  under  the  control  of  the 
data  processing  installation. 

In  some  cases  only  a  few  days  are  required  to  begin  to  develop 
(perhaps  crude)  answers. 

Since  significant  amounts  of  machine  resources  are  not  needed, 
very  valuable  interactive  "what  if"  exercises  are  possible. 

Non-IBM  hardware  can  be  modeled. 

Much  of  the  data  collection  to  develop  the  baseline  model  can  be 
automated  in  MVS  installations. 

Best/I  has  been  used  in  this  discussion  because  it  is  the  most  established 
package  available  for  acquisition  (and  judging  from  its  sales  growth  and 
customer  list,  it  has  found  good  acceptance). 
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Another  product,  Questor,  with  somewhat  similar  characteristics  has 
more  recently  been  marketed  by  Boole  and  Babbage.  (The  software  was 
developed  by  Performance  Systems,  Inc.,  Rockville,  MD,  which  also 
developed  the  simulation  package  SCERT.) 

•  It  should  be  reemphasized  that  as  attractive  and  valuable  as  such  software 
tools  can  potentially  be,  they  are  processing  only  numeric  assumptions  and 
estimates. 

To  be  adequate  predictors,  these  estimates  must  be  founded  on  business 
realities  and  a  correct  description  of  current  operations;  e.g.,  the 
complexities  and  ambiguities  illustrated  in  Exhibits  111-10  through  111-13 
must  be  reflected  in  both  the  baseline  case  and  estimates  for  the 
future. 

One  of  the  chief  virtues  of  simulation  and  modeling  is  to  allow  planners 
to  identify  variables  that  have  the  greatest  effect  on  capacity  needs; 
e.g.,  for  some  application/hardware  settings  a  20%  increase  in  applica- 
tion complexity  may  have  more  impact  than  a  20%  increase  in  volume 
(while  other  situations  may  be  vice  versa). 

Identifying  sensitivities  should  be  one  of  the  chief  goals  in 
planning  of  all  kinds. 
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THE  CURRENT  STATUS  OF  PERFORMANCE 
MEASUREMENT   AND   CAPACITY  PLANNING 


THE  CURRENT  STATUS  OF  PERFORMANCE  MEASUREMENT 
AND  CAPACITY  PLANNING 


This  chapter  will  describe  what  is  now  happening  in  the  field  of  performance 
measurement  and  capacity  planning.  Comparisons  will  be  made  between  the 
current  actuality  and  the  conceptual  framework  established  in  the  previous 
chapter.  Many  of  the  findings  and  conclusions  are  based  on  the  interviews 
which  INPUT  conducted  with  MIS  management. 

A  key  factor  is  the  growing  importance  of  performance  measurement  and 
capacity  planning  seen  by  respondents,  compared  to  three  years  ago,  as  shown 
in  Exhibit  IV- 1. 

The  increased  importance  extends  across  all  sizes  of  installation, 
although  larger  installations  appear  to  attach  somewhat  more  impor- 
tance to  these  functions. 

Even  with  this  sizable  increase  in  importance,  the  functions  are  still 
perceived  to  be  only  of  medium  importance. 

This  rating  is  especially  notable  since  the  respondents  usually  had  a 
direct  connection  with  performance  measurement  and  capacity 
planning,  and  always  had  a  high  level  of  interest.  However,  their 
ratings  appeared  to  be  a  very  realistic  reflection  of  the  level  of  accep- 
tance elsewhere  in  the  organization. 
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EXHIBIT  IV-1 


IMPORTANCE  OF  PERFORMANCE  MEASUREMENT  AND 
CAPACITY  PLANNING  AS  SEEN  BY  MIS  MANAGERS: 

1981  AND  1978 


INSTALLATION 
SIZE 


•   LESS  THAN  A  3033* 


•    3033  OR  LARGER* 


•  TOTAL 


0  1  2  3  4  5 

LOW  MEDIUM  HIGH 

IMPORTANCE 

NUMBER  OF  RESPONDENTS  =  34 

*  OR  EQUIVALENT,  SIZED  BY  CPU  RATINGS  IN  THOUSANDS  OF  OPERATIONS  PER  SECOND  (KOPS) 


-48  - 

©1981  by  INPUT.  Reproduction  Prohibited. 


•  There  do  not  appear  to  be  clear  patterns  in  the  type  of  company  that  places 
the  most  (or  least)  emphasis  on  performance  management  and  capacity 
planning. 

Several  companies  which  themselves  offer  data  processing-based 
services  had  until  recently  placed  very  little  importance  on  perform- 
ance measurement  and  capacity  planning. 

Overall  financial  resources  of  the  company  involved  may  play  a  role. 
Financially  bouyant  companies  appear  to  be  putting  greater  resources 
into  these  functions. 

If  any  pattern  emerged  it  was  an  unquantif iable  one  having  to  do  with 
corporate  and  personal  styles. 

Some  companies  plan  and  believe  in  planning,  others  don't  or 
can't. 

Similarly,  some  individuals  within  a  company  (especially  within  a 
data  processing  organization)  strongly  believe  in  measurement 
and  planning.  However,  the  corollary  is  that  when  such  people 
leave  their  position,  the  impetus  for  measurement  and  planning 
may  die  out. 

A.       ORGANIZATIONAL  ISSUES 

•  No  single  approach  is  taken  to  organize  performance  measurement  and 
capacity  planning.  However,  the  relationships  shown  in  Exhibit  IV-2  reflect 
the  typical  approach. 
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EXHIBIT  IV-2 


CAPACITY  PLANNING  ORGANIZATION: 
TYPICAL  CURRENT  STRUCTURE 


VP  -  MIS 

DIRECTOR  - 
OPERATIONS 


MANAGER  -  PERFORMANCE 
MEASUREMENT  AND 
CAPACITY  PLANNING 


DIRECTOR  - 
DEVELOPMENT 
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Justification  for  this  approach  is  tool-driven:  measurement  tools  are 
largely  in  the  domain  of  systems  programming,  therefore  the  function 
that  uses  these  tools  should  be  also. 

An  alternative  organization  used  by  several  installations  is  to  include  capacity 
planning  within  their  MIS  planning  unit.  (Of  course,  many  installations  have 
only  a  vestigial  planning  function.) 

Of  perhaps  greater  near-term  importance  than  the  organizational  placement  is 
the  number  of  people  assigned  to  performance  measurement  and  capacity 
planning.  Often  there  is  an  insufficient  level  of  personnel  resources  assigned 
to  these  functions. 

This  is  an  inescapable  problem  in  all  but  the  largest  installations.  Exhibit  IV-3 
shows  the  average  size  of  staff  assigned  to  performance  measurement  and 
capacity  planning  by  installation  size. 

On  the  average,  larger  installations  had  three  times  as  many  personnel 
resources  assigned  to  these  functions.  But: 

Two-thirds  of  the  respondents  had  fewer  than  two  people 
assigned  to  performance  measurement  and  capacity  planning. 

Virtually  none  of  the  smaller  installations  had  two  or  more 
people  assigned  to  the  functions. 

Over  half  the  smaller  installations  had  no  full-time  staff 
assigned  to  the  functions. 

What  is  the  minimum  effective  number  of  staff  required  for  effective 
performance  measurement  and  capacity  planning? 

An  experienced  capacity  planner  within  IBM  has  written,  for  example, 
that   having   two  people  start   the   performance   measurement  and 
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EXHIBIT  IV-3 


STAFF  ASSIGNED  TO  PERFORMANCE 
MEASUREMENT  AND  CAPACITY  PLANNING 


PERCENT 

OF  INSTAL- 

PERCENT 

LATIONS  WITH 

AVERAGE 

STAFF 

WITH  NO 

TWO  OR  MORE 

INSTALLATION 

STAFF 

SIZE 

ASSIGNED 

FULL-TIME 

SIZE 

SIZE 

RANGE 

STAFF 

EQUIVALENTS 

LESS  THAN  A  3033* 
(n  =  15) 

0.7 

0-3 

53% 

13% 

3033  OR  LARGER* 
(n  =  14) 

2.7 

0-8 

21 

55 

TOTAL  (n  =  34) 

1.6 

0-8 

35% 

35% 

*  OR  EQUIVALENT,  SIZED  BY  CPU  KOPS  RATING 
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capacity  planning  process  "would  be  the  most  reasonable  requirement" 
(Bronner,  IBM  Systems  Journal,  Vol.  19,  No.  I). 


Basic  economics  and  priorities  often  dictate  a  more  modest  approach. 
However,  installations  which  have  only  a  single  person  assigned  to 
performance  measurement  and  capacity  planning  will  not  have  a  very 
effective  operation: 

During  vacations  and  other  absences  (e.g.,  on  crash  projects)  the 
functions  will  not  be  covered. 

If  and  when  the  single  person  leaves  for  another  job  (whether 
within  or  outside  the  same  company),  performance  measurement 
and  planning  will  essentially  have  to  start  over  again.  This  was 
observed  in  several  companies  interviewed. 

Performance  measurement  and  capacity  planning  call  for  a  wide 
range  of  skills.  It  is  unlikely  that  all  the  skills  will  be  found  to 
the  same  degree  in  the  same  person.  It  is  even  probable  that  a 
person  will  be  deficient  in  critical  areas  (e.g.,  a  person  may  be 
basically  a  systems  programmer  and  not  be  applications- 
oriented). 

Perhaps  most  important,  having  two  or  more  persons  working  on 
performance  measurement  and  capacity  planning  will  provide  for 
a  diversity  of  points  of  view,  constructive  criticism,  and  error 
avoidance.  A  process  of  internal  review  is  most  important  if  the 
resulting  technical  proposals  are  to  be  well  thought  out  and 
grounded  in  reality.  Otherwise,  performance  tuning  and  capacity 
planning  will  be  at  best  unbalanced  and  all  too  often  subject  to 
ill-judged  enthusiasms,  as  well  as  just  plain  mistakes. 
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Judged  by  this  reasoning,  most  companies  are  currently  condemned  to  an 
ineffective  program  of  performance  measurement  and  capacity  planning  due 
to  a  lack  of  resources. 


Perhaps  part  of  the  reason  for  the  relatively  low  importance  of 
performance  measurement  and  capacity  planning  in  some  companies  is 
that  the  program  does  not  produce  results  sufficient  to  justify  itself. 
(In  at  least  a  few  of  the  companies  interviewed,  the  low  importance 
given  the  functions  was  a  direct  cause  of  their  lack  of  resources.) 

•  There  are  indications  that  more  resources  are  now  being  allocated  to  these 
functions. 

In  several  companies,  the  functions  had  just  been  initiated  or  greatly 
expanded. 

In  others,  these  were  plans  to  expand  them. 

In  no  company  were  there  indications  that  the  functions  were  to  be 
reduced  in  scope. 

B.       PERFORMANCE  MEASUREMENT 

I.        USE  OF  JOB  ACCOUNTING  REPORTS 

•  Over  70%  of  installations  interviewed  use  a  job  accounting  system,  with 
smaller  installations  somewhat  less  prone  to  use  one,  as  shown  in  Exhibit  IV-4. 

•  About  as  many  installations  use  a  "homegrown"  system  as  use  a  commercial 
package,  as  shown  in  Exhibit  IV-5. 

One-seventh  of  those  interviewed  use  a  manual  system. 
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EXHIBIT  IV-U 


USE  OF  JOB  ACCOUNTING  SYSTEMS 
BY  INSTALLATION  SIZE 


INSTALLATION  SIZE 


LESS  THAN  A  3033 


•  3033  OR  LARGER 


•  TOTAL 


1001 


NUMBER  OF  RESPONDENTS  =  34 


PERCENT  USING  A  JOB 
ACCOUNTING  SYSTEM 
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EXHIBIT  IV-5 


JOB  ACCOUNTING  METHODOLOGIES 


METHODOLOGY 


MANUAL  SYSTEM 


HOMEGROWN"  SYSTEM 


COMMERCIAL  PACKAGE 


NONE 


20 


HO 


60 


80 


100 


PERCENT  OF  RESPONDENTS 


NUMBER  OF  RESPONDENTS  =  34 
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The  particular  interest  in  job  accounting  from  the  standpoint  of  this  study  is 
because  some  firms  view  job  accounting  as  a  form  of  performance  measure- 
ment. (Note  in  the  following  subsection  that  a  significant  number  of  firms' 
only  source  of  performance  measurement  is  job  accounting  data.) 

USE  OF  PERFORMANCE  MEASUREMENT  TOOLS 

Almost  all  installations  use  at  least  one  type  of  measurement  tool,  as  shown  in 
Exhibit  IV-6.  Two-thirds  use  more  than  one  tool. 

The  most  common  single  approach  is  to  use  RMF/SMF  data  directly,  as  shown 
in  Exhibit  IV-7,  although  software  packages  (job  accounting  package,  monitors, 
and  other  software)  are  used  more  if  looked  at  as  a  unit. 

Hardware  monitors  are  in  place  in  a  small  minority  of  installations 
interviewed. 

A  small  minority  also  use  manual  methods. 

It  is  difficult  to  see  much  of  a  pattern  in  the  use  of  measurement  tools.  In 
Exhibit  IV-8,  sites  are  arranged  in  ascending  order  of  size  (ranked  by  CPU 
KOPS  rating)  and  the  major  categories  of  measurement  methodologies  are 
listed  for  each  site. 

There  do  not  appear  to  be  strong  relationships  between  types  of  tools 
used,  installation  size,  or  the  size  of  the  performance  measurement  and 
capacity  planning  staff,  except  that  the  biggest  sites  tend  to  perform 
more  functions  themselves. 

The  concurrent  methodologies  used  are  summarized  in  Exhibit  IV-9. 
Again,  no  particular  pattern  of  interrelationships  between  methodol- 
ogies emerges. 
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EXHIBIT  IV-6 


NUMBER  OF  PERFORMANCE  MEASUREMENT 
TOOLS  USED  PER  INSTALLATION 


NUMBER  OF  PERFOR- 
MANCE TOOLS  USED 


FOUR 


•  THREE 


•  TWO 


•  ONE 


NONE 


10 


20 


30 


40 


50 


PERCENT  OF  INSTALLATIONS 


NUMBER  OF  RESPONDENTS  =  34 
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EXHIBIT  IV-7 


MEASUREMENT 
TOOLS  

MANUAL  METHODS 


HARDWARE 
MONITORS 


•  ACCOUNTING 
PACKAGES 


SMF  DATA 


•  SOFTWARE 
MONITORS 


•OTHER  SOFTWARE 
PACKAGES 


•  RMF  DATA 


RELATIVE  USE  OF  DIFFERENT 
MEASUREMENT  TOOLS 


NUMBER  OF  RESPONDENTS  =  34 


10 


20 


30 


40 


AMOUNT  OF  TOOL  USE  (AS  A 
PERCENT  OF  ALL  TOOLS) 
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EXHIBIT  IV-8 
MEASUREMENT  METHODOLOGIES  EMPLOYED 


STAFF 
SIZE* 

JOB  AC- 
COUNTING 
PACK- 
AGE** 

SMF 
DATA 

RMF 
DATA 

STATIS- 
TICAL 
FRONT 
END  TO 
SMF/RMF 

SOFT- 
WARE 
MONITOR 

OTHER 
SOFT- 
WARE 

HARD- 
WARE 
MONITOR 

MANUAL 
TECH- 
NIQUE 

TOTAL 
TOOLS 
USED 

1 

X 

— 

— 

— 

— 

X 

— 

— 

2 

0 

X 

1 

0 
0 
0 
0 
3 
0 

— 

— 

- 

— 

— 

X 

- 

V 

1 

1 
2 
0 
2 
1 





X 

X 

— 

— 

— 

A 

- 



- 


V 
A 

X 

v 

A 

— 



— 


— 
_ 

- 

— 

1 

0 

X 
X 

— 
— 

— 
X 

— 

X 

— 
— 

— 
X 

— 
— 

— 
— 

1 
4 

1 

0 
3 

- 

X 

— 

— 

— 

X 

— 

— 

2 
1 
4 

— 

- 

— 
X 

— 
X 

— 
X 

— 
— 

— 
X 

X 
— 

— 
- 

1 
1 
0 

X 

X 

X 

3 
1 
2 

— 

X 
— 

— 

— 

X 

— 

— 

X 

4 

— 

— 

— 

— 

— 

— 

— 

X 

1 

1 

— 

— 

X 

— 

X 

— 

— 

— 

2 

0 

— 

X 

X 

X 

X 

— 

— 

— 

4 

2 

— 

X 

— 

— 

— 

— 

X 

— 

2 

3 
1 
0 

— 

X 

X 
X 
X 

X 

X 

— 

— 

— 

4 
1 

3 

X 

— 
- 

— 
- 

- 
X 

- 
- 

- 
— 

— 
— 

0.5 

v 
A 

v 
A 

2 

2 

— 

_ 

— 

X 

— 

X 

— 

2 

1 

X 

X 

X 

— 

X 

- 

- 

— 

4 

5.5 

— 

X 

— 

X 

X 

— 

3 

1 

V 
A 

X 

L 

4 

X 

X 

X 

3 

3 

X 

X 

2 

2 
0 
8 

X 

X 

2 
0 
2 

X 

X 

5 

X 

X 

X 

3 

TOTALS 

5 

12 

18 

7 

13 

8 

4 

3 

70 

♦NUMBER  OF  PEOPLE  ASSIGNED  TO  PERFORMANCE  MEASUREMENT  AND  CAPACITY  PLANNING  FUNCTIONS. 
**IF  THE  JOB  ACCOUNTING  PACKAGE  IS  USED  FOR  PERFORMANCE  MEASUREMENT. 

NOTE:  SITES  ARRANGED  IN  ORDER  OF  INCREASING  SIZE,  AS  MEASURED  IN  CPU  RATING  IN  THOUSANDS 
OF  OPERATIONS  PER  SECOND  (KOPS).    EACH  LINE  REPRESENTS  ONE  SITE. 
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EXHIBIT  IV-9 


CONCURRENT  MEASUREMENT  METHODOLOGIES 


• 

12 

SMF  USERS 

FOUR  USE  A  STATISTICAL  "FRONT  END11 

SEVEN  USE  RMF 

NINE  USE  ANOTHER  SOFTWARE  PACKAGE  OR 

HARDWARE  MONITOR 

• 

18 

RMF  USERS 

SIX  USE  A  STATISTICAL  "FRONT  END" 

SEVEN  USE  SMF 

12  USE  ANOTHER  SOFTWARE  PACKAGE  OR 

HARDWARE  MONITOR 

• 

FIVE  ACCOUNTING  PACKAGE  USERS 

FOUR  USE  ANOTHER  METHODOLOGY  AS  WELL 

• 

13 

SOFTWARE  MONITOR  USERS 

ALL  13  USE  ANOTHER  METHODOLOGY  AS  WELL 

• 

EIGHT  USERS  OF  OTHER  SOFTWARE,  (e.g.,  SCHEDULERS, 

REPORTING  PACKAGES) 

SIX  USE  ANOTHER  METHODOLOGY  AS  WELL 

• 

FOUR  HARDWARE  MONITOR  USERS 

THREE  USE  ANOTHER  METHODOLOGY  AS  WELL 

• 

THREE  USERS  OF  MANUAL  ANALYSIS  METHODS 

ONE  IN  CONJUNCTION  WITH  A  SOFTWARE  PACKAGE 
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To  a  certain  extent,  the  mixture  of  tools  reflects  tools  "inherited"  from 
departed  measurement  specialists,  where  the  incoming  specialist 
prefers  to  use  a  different  tool. 

Appendix  D  lists  a  number  of  performance  measurement  software  tools. 

RESULTS  OF  PERFORMANCE  MEASUREMENT 

About  two-thirds  of  installations  felt  that  they  had  already  experienced 
improvements  in  throughput  and  response  time  by  system  tuning  as  a  result  of 
performance  measurement,  as  shown  in  Exhibit  IV- 10.  One-third  of  installa- 
tions (in  both  areas)  also  anticipated  future  improvements. 

The  average  improvement  for  those  who  could  quantify  improvements  was  said 
to  be  around  50%  for  both  throughput  and  response  time,  with  about  one- 
seventh  of  installations  seeing  improvements  of  over  100%,  as  shown  in  Exhibit 
IV-I  I. 

It  is  somewhat  disturbing  that  one-quarter  of  the  installations  could 
offer  no  quantification,  given  that  the  benefit  was  supposed  to  be  a 
direct  outcome  of  performance  measurement. 

While  the  improvements  reported  are  in  line  with  some  of  those  reported  in 
the  literature,  it  is  INPUT'S  opinion  that  these  figures  should  be  treated  with 
caution: 

Many  of  the  respondents  gave  figures  described  as  "estimates";  in  other 
cases,  the  manner  in  which  replies  were  given  implied  that  the  figures 
were  estimates. 

In  some  interviews,  the  bases  for  the  numbers  were  probed.  They 
appeared  in  most  cases  to  be  the  result  of  isolated  measurements  that 
may  or  may  not  have  been  representative.  There  were  very  few 
installations  with  reliable,   comprehensive  baseline   data.  Several 
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EXHIBIT  IV-10 


RESPONDENTS'  REPORTED  AND  ANTICIPATED 

IMPROVEMENTS  IN  THROUGHPUT  AND  RESPONSE  TIME 


THROUGHPUT 
IMPROVEMENTS 


NOW 


FUTURE 


RESPONSE  TIME 
IMPROVEMENTS 


NOW 


FUTURE 


100% 


PERCENT  OF  INSTALLATIONS 


NUMBER  OF  RESPONDENTS  =  34 
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EXHIBIT  IV-11 


EXTENT  OF  IMPROVEMENT  IN  THROUGHPUT 
AND  RESPONSE  TIME 


PERCENT  OF 
THAT  SAW 

INSTALLATIONS 
IMPROVEMENT 

PERCENT 
IMPROVEMENT 

THROUGHPUT 

RESPONSE 
TIME 

100%  AND  OVER 

16% 

14% 

50-99% 

8 

29 

49%  AND  UNDER 

52 

34 

AMOUNT  UNDETERMINED 

24 

24 

TOTAL 

100% 

100% 

NUMBER  OF  RESPONDENTS  =  34 
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respondents  commented  on  the  difficulty  of  tracking  and  characterizing 
a  "moving  target." 

These  difficulties  were  confirmed  by  several  performance  software 
vendors;  this  is  significant,  since  it  is  certainly  in  the  vendors'  interest 
to  be  able  to  document  case  histories  of  successful  uses  of  performance 
software. 

•  While  performance  measurement  software  is  considered  quite  important,  the 
satisfaction  over  its  use  is  lower  than  its  importance,  as  shown  in  Exhibit  IV- 
12. 

In  the  case  of  larger  installations,  importance  is  seen  as  higher  and 
satisfaction  as  lower  than  in  smaller  installations. 

•  A  fair  conclusion  is  that  improvements  in  performance  measurement  software 
are  still  required. 

4.        PERFORMANCE  MEASUREMENT  PROBLEMS 

•  The  lack  of  focus  on  the  use  of  measurement  tools,  described  earlier,  is 
consistent  with  the  dissatisfaction  and  with  the  more  basic  problems  observed 
during  the  interviews  (and  confirmed  in  discussions  with  vendors). 

Most  firms  do  not  have  a  systematic  approach  to  performance  measure- 
ment; the  users  of  tools  are  not  sure  what  they  are  to  be  measuring, 
consequently  they  are  often  unsure  of  the  right  tool  to  use. 

Many  performance  measurement  specialists  do  not  have  a  broad  view  of 
the  field  and  some  have  not  yet  received  enough  training  to  select  the 
right  tool. 
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EXHIBIT  IV-12 


JOB  ACCOUNTING  AND  PERFORMANCE 
PACKAGE  -  IMPORTANCE  AND  SATISFACTION 


INSTALLATION  SIZE 


LESS 
THAN  A 

3033* 


3033 
OR 

LARGER* 


TOTAL 


IMPORTANCE 


SATISFACTION 


IMPORTANCE 


SATISFACTION 


IMPORTANCE 


SATISFACTION 


1 

LOW 


2  3 

MEDIUM 


5 

HIG 


NUMBER  OF  RESPONDENTS  =  31 

*OR  EQUIVALENT,  SIZED  BY  CPU  KOPS  RATING 
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The  organizations  with  no  performance  measurement  specialists  are  in  a 
worse  position  than  those  where  the  specialists  have  received  even 
inadequate  training. 


In  both  cases,  inappropriate  tools  may  very  well  be  selected.  For 
example,  installations  that  are  relying  primarily  on  job 
accounting  data  for  performance  measurement  (one-quarter 
of  those  interviewed)  will  be  missing  some  very  important 
measurements. 

Even  where  valid  data  are  being  collected,  a  recurring  complaint  is  that  the 
data  are  not  being  used  effectively: 

"We  generate  tons  of  data,  but  don't  know  what  it  means."  One 
installation  had  accumulated  150  reels  of  tape  containing  SMF/RMF 
data  and  didn't  know  how  to  proceed! 

Variations  on  the  following  reply  were  received  from  several  respon- 
dents when  asked  of  their  plans  for  acquiring  additional  software:  "We 
know  we  aren't  using  our  present  tools  to  their  fullest." 

A  warning  flag,  at  the  very  least,  should  be  raised  over  the  fact  that  so  many 
companies  (over  half  those  interviewed)  process  their  own  RMF  data. 

This  is  not  unexpected,  given  the  predilection  of  many  systems  pro- 
grammers to  do  things  their  own  way.  However,  systems  programmers 
have  a  much  higher  tolerance  for  detail  than  management. 

There  is  also  an  element  of  reinventing  the  wheel.  Software  packages 
at  least  have  the  virtue  of  concentrating  and  focusing  their  reporting 
considerably  (although  more  progress  could  be  made). 

It  is  striking  that  the  dissatisfaction  is  relatively  high  in  the  installations  that, 
by  and  large,  have  the  more  adequate  staffs  and  resources.     In  INPUT'S 
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opinion,  this  reflects  the  fact  that  they  have  acquired  enough  knowledge  to  be 
aware  of  how  much  they  still  do  not  know. 

C.       CAPACITY  PLANNING 

I .       THE  PLANNING  PROCESS 

•  Over  two-thirds  of  installations  interviewed  have  workload  targets,  as  shown 
in  the  top  line  of  Exhibit  IV- 1 3.  Almost  as  many  try  to  project  five  years  into 
the  future  as  try  to  project  six  months. 

Many  of  those  who  had  five-year  projections  expressed  great  doubt  as 
to  the  worth  of  such  an  exercise,  but  explained  that  it  was  necessary  as 
part  of  their  capital  equipment  budgeting  process. 

Almost  two-thirds  of  those  with  targets  said  that  they  met  their  six- 
month  targets  and  almost  half  said  that  they  met  the  five-year  targets, 
as  shown  in  the  second  line  of  Exhibit  IV- 1 3. 

•  It  is  fair  to  say  that  in  a  number  of  cases  the  stated  success  in  meeting  targets 
represented  as  much  a  hope  as  a  reality. 

Two-thirds  said  that  they  had  very  stringent  targets;  i.e.,  over  90% 
accuracy  in  projecting  targets  five  years  into  the  future,  as  shown  in 
the  middle  portion  of  Exhibit  IV-13. 

It  was  apparent  that  some  respondents  really  had  qualitative  measures 
of  success  in  mind;  e.g.,  "I'm  not  aware  of  any  disasters  and  I  haven't 
lost  my  job,  so  we  must  be  meeting  targets"  (implicit  in  several 
interviews). 
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EXHIBIT  IV-13 


WORKLOAD  PLANNING  TARGETS 
AND  SUCCESS  IN  MEETING  THEM 
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A  few  respondents  made  obviously  contradictory  responses  in  the  course 
of  the  interview;  e.g.,  one  installation  that  claimed  success  in  meeting 
high  accuracy  targets  discussed  how  one  large  machine  had  been  sitting 
virtually  idle  for  an  extended  period  of  time.  It  was  obvious  in  that 
case  that  the  respondent  was  excluding  from  accuracy  assessments 
events  over  which,  in  his  opinion,  his  department  had  no  direct  control. 

Most  importantly,  as  discussed  in  the  previous  section,  most  installa- 
tions do  not  have  accurate  short-term  (not  to  speak  of  long-term) 
baseline  data  on  which  to  make  judgments  of  workloads. 

Consistent  workload  and  capacity  baseline  measurement  are 
certainly  very  difficult  to  maintain  over  a  multiyear  period, 
given  ongoing  changes  in  machine  configurations  and  qualitative 
changes  in  the  nature  of  the  workload. 

•  It  is  quite  likely  that  many  installations  still  have  an  implicitly  batch-oriented 
mental  picture  of  workload  and  capacity.  Exhibit  IV- 14  shows  two  ways  of 
viewing  capacity. 

View  "A"  is: 

Inward-looking. 

Process-oriented. 

Defined  by  computer  resources. 
View  "B"  is: 

Outward-looking. 

Results-oriented. 

Defined  by  user  needs. 
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EXHIBIT  IV-14 


TWO  VIEWS  OF  CAPACITY 


100% 


CAPACITY  DANGER  ZONE 


X  PERCENT  OF  "CAPACITY" 


RESPONSE  TIME 


TIME   ► 
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However,  view  "A"  is  still  respected;  in  the  literature  as  well  as  in  INPUT'S 
interviews,  CPU  time  is  still  often  used  as  the  point  of  reference  because: 

It  is  relatively  easy  to  measure. 

It  can  be  equated  from  machine  to  machine  (although  perhaps  not  as 
easily  as  sometimes  believed). 

It  is  a  traditional  measure. 

In  deference  to  modernity,  rather  than  a  90%  CPU  overall  target  being 
established  for  planning  purposes,  some  installations  now  have  a  30%  (or  so) 
level  established  for  the  prime  shift. 

However,  such  a  surrogate  does  not  take  into  account  the  possibility 
that  interactive  system  performance  can  be  degraded  for  a  variety  of 
reasons  other  than  CPU  constraints. 

A  handful  of  installations  interviewed  now  view  response  time  as  the  critical 
measure  on  which  they  were  basing  their  planning  efforts.  Several  others  were 
moving  in  that  direction. 

Success  will  probably  be  limited,  however,  until  baseline  measurement 
is  improved. 

Two-thirds  of  the  installations  reported  that  estimates  of  user  demand  are 
part  of  the  capacity  planning  process. 

However,  in  most  cases  this  consists  only  of  estimating  transaction 
volume  increases. 
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CAPACITY  PLANNING  TOOLS 


Few  installations  are  using  special  tools  for  capacity  planning,  in  contrast  to 
performance  measurement,  as  shown  in  Exhibit  IV- 1 5. 

Others  are  in  the  experimentation  stage,  but  there  is  a  general  dearth  of 
knowledge  on  the  subject. 

Two-thirds  were  unable  to  name  any  (or  any  additional)  capacity 
planning  tools,  as  shown  in  Exhibit  IV—  1 6. 

This  is  partly  explained  by  the  relatively  few  tools  now  available  and 
their  relatively  low  penetration. 

It  is  doubtful  if  capacity  planning  can  be  effective  without  the  use  of 
specialized  capacity  planning  tools. 

Statistical  trending  methods  based  on  past  history,  while  straight- 
forward, will  not  reflect  important  realities. 

Manual  methods  will  be  too  slow  and  imprecise  and  will  not  permit  any 
significant  amount  of  sensitivity  analysis. 
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EXHIBIT  IV-15 


CAPACITY  PLANNING  TOOLS  NOW  EMPLOYED 


SAME  TOOLS  USED 
FOR  PERFORMANCE 
MEASUREMENT 


ADDITIONAL  SOFT- 
WARE TOOLS 


NO  TOOLS  USED 
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NUMBER  OF  RESPONDENTS  =  34 
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KNOWLEDGE  OF  ADDITIONAL 
CAPACITY  PLANNING  TOOLS  AVAILABLE 
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V  TRENDS 


Two  trends  are  important  for  MIS  management  to  keep  informed  of: 


Trends  within,  or  perceived  by,  data  processing  installations. 


Trends  within  the  vendor  community. 


A. 


TRENDS  IN  DATA  PROCESSING  INSTALLATIONS 


One  important  way  of  predicting  trends  is  to  identify  unmet  needs. 


Over  half  the  installations  interviewed  saw  their  biggest  need  being 
improved  methodologies  for  measurement  and  planning,  as  shown  in 
Exhibit  V-l. 

The  greater  need  was  seen  in  performance  measurement;  this  is 
reasonable,  given  the  many  current  problems  in  this  area  and 
that  good  performance  measurement  is  the  necessary  precondi- 
tion to  capacity  planning. 

Improved  performance/tuning  and  trained  personnel  were  also  seen  as 
key  needs  by  15%  of  those  interviewed. 
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EXHIBIT  V-1 


LARGEST  UNMET  NEED 


IMPROVED  PERFOR- 
MANCE MEASURE- 
MENT METHODOLOGY 
AND  DATA 

IMPROVED  CAPACITY 
PLANNING  AND 
FORECASTING 


IMPROVED  PERFOR- 
MANCE AND  TUNING 


TRAINED  PERSONNEL 


NONE/DON'T  KNOW 


PERCENT  OF  INSTALLATIONS 


NUMBER  OF  RESPONDENTS  =  34 
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Exhibit  V-l  should  be  viewed  as  a  good  overall  list  of  priorities.  It  should  be 
kept  in  mind  that  a|[  these  items  are  important  issues  (and  were  important  in 
the  minds  of  most  respondents). 

A  majority  of  installations  interviewed  saw  an  increase  in  importance  for 
performance  measurement  and  capacity  planning  as  being  the  most  significant 
trend  over  the  next  five  years  in  these  areas,  as  shown  in  Exhibit  V-2. 

Another  quarter  saw  primarily  technical  trends. 

A  very  small  number  saw  closer  involvement  in  business  planning  as 
being  of  primary  importance. 

It  is  quite  likely  that  this  response  is  so  prevalent  because: 

There  is  a  sense  of  increased  interest  and  activity  in  the  general  area 
that  has  not  yet  reached  its  peak. 

In  many  cases,  there  is  both  a  lack  of  reflection  on  specific  causative 
factors  and  a  lack  of  knowledge  of  the  external  environment. 

One  issue  that  the  study  explored  in  some  depth  was  whether  falling  hardware 
prices  would  lessen  the  need  for  capacity  planning  (and  performance  measure- 
ment, for  that  matter). 

It  is  quite  significant  that  almost  half  the  installations  saw  falling  hardware 
prices  requiring  more  in  the  way  of  capacity  planning,  as  shown  in  Exhibit  V-3. 
The  reasons  given  fall  into  two  categories,  as  shown  in  Exhibit  V-4. 

Increases  in  software  prices  will  at  least  partially  offset  falling 
hardware  prices. 
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EXHIBIT  V-2 


TRENDS  FORESEEN  IN  PERFORMANCE  MEASUREMENT  AND 
CAPACITY  PLANNING  IN  THE  NEXT  FIVE  YEARS 


WILL  INCREASE  IN 
IMPORTANCE  AND 
USE 

WILL  HAVE  IMPROVED 
TECHNICAL 
APPROACHES 


WILL  BE  BUILT  INTO 
HARDWARE 


WILL  BECOME  MORE 
INVOLVED  IN  BUSI 
NESS  PLANNING 


DON'T  KNOW 


100 


NUMBER  OF  RESPONDENTS  =  35 
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EXHIBIT  V-3 


IMPACT  FORESEEN  OF  FALLING  HARDWARE 
PRICES  ON  CAPACITY  PLANNING 


MORE  PLANNING 
NEEDED 


NO  IMPACT  FORESEEN 


LESS  PLANNING 
NEEDED 


DON'T  KNOW/OTHER 


0  10  20  30  40 


NUMBER  OF  RESPONDENTS  =  35 
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EXHIBIT  V-4 


REASONS  WHY  FALLING  HARDWARE 
PRICES  WILL  REQUIRE  MORE  PLANNING 


OFFSETTING  SOFT- 
WARE PRICE 
INCREASES 

HARDWARE  USE 
WILL  BE  EN- 
COURAGED 


NO  SPECIFIC 
REASON  GIVEN 


10 


20 


30 


40 


50% 


NUMBER  OF  RESPONDENTS  =  35 
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Falling  hardware  prices  will  stimulate  computer  use,  causing  total 
expenses  to  continue  to  rise.  These  relationships  are  shown  in  Exhibit 
V-5. 


Other  issues  explored  were  standalone  user-operated  systems  and  distributed 
data  processing.  The  responses  were  similar  in  both  areas,  as  shown  in  Exhibit 
V-6. 

About  one-quarter  of  installations  saw  planning  becoming  more  complex 
for  both  areas. 

About  one-third  saw  use  increasing,  with  the  effects  on  planning 
unclear  (half  saw  DDP  in  this  light). 

It  would  probably  be  fair  to  characterize  this  group  as  hoping 
that  there  will  be  no  impact  from  these  -  but  having  to  admit 
that  there  will  in  fact  possibly  be  a  substantial  impact. 

INPUT  received  a  strong  impression  during  the  interview  process  that  most 
installations  were  not  (and  did  not  feel)  ready  to  begin  measuring  and  planning 
for  a  dispersed  data  processing  environment  (whether  standalone  or 
distributed). 

However,  the  interplay  between  standalone  and  central  data  processing 
functions  will  become  much  more  important  -  and  complex  -  in  the  near 
future. 

For  example,  INPUT'S  recent  study,  Personal  Computers  in  Large 
Companies,  forecasts  a  sevenfold  rise  in  the  use  of  personal 
computers  by  Fortune  500  companies  by  1985  -  often  in  critical 
application  areas. 
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EXHIBIT  V-5 


DATA  CENTER  COSTS:  SCHEMATIC 


TIME 
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EXHIBIT  V-6 


IMPACT  OF  STANDALONE  USER-OPERATED  SYSTEMS  AND 
DISTRIBUTED  DATA  PROCESSING  ON  CAPACITY  PLANNING 


NO  IMPACT 


WILL  SEE  INCREASED 
USE  -  EFFECTS 
UNCLEAR 


PLANNING  WILL  BE 
MORE  COMPLEX 


DON'T  KNOW 


10 


20 


30 


40 


50% 


STANDALONE 
DDP 
|    |  TOTAL 

NUMBER  OF  RESPONDENTS  =  35 
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VENDOR  TRENDS 


Vendors  realize  that  their  own  products  are  part  of  the  problem. 

There  are  now  a  large  number  of  software  products  that  generate  a  profusion 
of  data.  In  the  future,  software  products  should  better  serve  data  processing 
installations  in  a  number  of  ways: 

Related  products  will  be  better  integrated  so  that  there  is  less  user 
intervention  and  routine  decision-making,  as  shown  in  Exhibit  V-7. 
Examples  of  this  trend  already  exist: 

The  related  products  offered  by  BGS  (a  monitor,  program 
resource  projector,  and  system  resource  projector). 

CMF  from  Boole  and  Babbage  (which  is  a  consolidation  of  several 
earlier  products). 

The  April  1980  SPARK  release  from  Burroughs. 

All  indications  are  that  this  trend  will  accelerate.  There  is  no 
reason,  for  example,  why  a  single,  modular  software  tool  could 
not  integrate  performance  measurement  and  reporting  functions 
with  capacity  planning. 

Future  software  should  be  able  to  take  corrective  action  by  itself 
(within  predetermined  bounds)  rather  than,  at  best,  issue  warnings,  as  is 
now  the  case. 

A  distinction  has  grown  up  between  batch  and  on-line  software 
monitors. 
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INTEGRATION  OF 
MEASUREMENT  AND  PLANNING 
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Batch  monitors  accumulate  log  data,  perhaps  daily,  and  produce 
reports  showing  the  contrasts  between  relatively  recent  and 
historic  performance. 

On-line  monitors  show,  via  a  CRT,  what  is  happening  at  a 
particular  instant,  but  do  not  compare  that  instant  to  historic 
data. 

Some  on-line  monitors  (e.g.,  ADR's  LOOK)  do  accumulate  data 
being  collected  for  later  batch-type  reporting. 

The  next  generation  of  monitors  would  allow  for  real-time 
interrogation  of  historic  files.  This  would  allow,  for  example, 
the  automatic  corrective  action,  described  above,  to  use  historic 
experience  as  one  of  the  control  parameters. 

In  the  future,  software  packages  will  also  provide  for  more  extensive 
focused  reporting  that  will  guide  users  in  establishing  more  effective 
exception  reporting. 

It  is  useful  to  view  software  tools  as  potentially  including  a  hierarchy  of 
functions,  as  shown  in  Exhibit  V-8. 

Generally  speaking,  the  batch  monitor  functions  are  toward  the  lower 
end  of  the  spectrum  (which  can  also  be  viewed  as  a  necessary 
foundation). 

The  higher  integrated  levels  are  still  in  the  process  of  development. 

Hardware  monitors  are  a  special  case.  They  offer  unique  functions,  but  at  the 
present  time  relatively  few  installations  have  progressed  far  enough  to  use 
them  effectively  or,  in  many  instances,  know  that  they  might  need  a  hardware 
monitor. 
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EXHIBIT  V-8 


HIERARCHY  OF  SOFTWARE 
TOOL  FUNCTIONS 


NOW  OFFERED  IN: 

LEVEL 

FUNCTION 

ON-LINE 
MONITORS 

BATCH 
MONITORS 

6 

AUTOMATIC  CORRECTIVE  ACTION 

5 

AUTOMATIC  WARNING  FOR  EX- 
CEEDED PARAMETERS 

SOME 

4 

EXCEPTION  REPORTING 

YES 

SOME 

3 

FOCUSED  REPORTING 

SOME 

YES 

2 

SUMMARIZED/ORGANIZED  DATA 

SOME 

YES 

1 

INDISCRIMINATE/UNSELECTIVE 
DATA  COLLECTION 

YES 
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As  knowledge  and  familiarity  with  software  monitors  build  up,  more 
installations  would  be  expected  to  explore  the  use  of  hardware 
monitors. 

An  intriguing  product  would  be  a  combined  hardware-software  monitor 
that  would  combine  the  advantages  of  both  types  of  monitoring  in  a 
single  package. 

•  Another  possibility  would  be  pulling  back  into  microcode  the  monitoring 
functions  that  are  now  performed  externally.  With  the  absorption  of  other 
systems  functions  into  firmware,  this  kind  of  transfer  of  functions  is  feasible. 

One  hardware  company  is  considering  offering  a  built-in  hardware 
monitor  as  part  of  an  approach  to  making  performance  measurement 
more  effective  and  easier  to  use. 

IBM  could  doubtless  offer  this  capability  if  it  was  judged  to  be  in  IBM's 
interest  to  do  so.  The  effects  on  other  performance  monitoring  vendors 
would  be  profound.  The  benefits  (if  any)  for  data  processing  installa- 
tions would  depend  on  the  way  in  which  the  facility  was  implemented 
and  how  much  room  was  left  for  other  vendors. 

If  IBM's  handling  of  similar  software  issues  is  any  guide,  it  is 
doubtful  that  there  will  be  any  rapid  changes. 

C.       CAPACITY  PLANNING  COMPLEXITIES 

•  The  previous  two  sections  described  many  positive  tendencies,  insofar  as  the 
capabilities  of  data  processing  installations  to  better  measure  performance 
and  plan  capacity  are  concerned. 
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•  However,  INPUT  expects  to  see  a  concomitant  increase  in  the  complexity  and 
difficulty  of  carrying  out  effective  capacity  planning.  Most  of  these  factors 
have  been  discussed  before  and  will  just  be  mentioned  here: 

Increased  use  and  importance  of  on-line  systems. 

Distributed  data  processing. 

User  standalone  systems. 

•  Another  development  that  only  a  few  installations  have  fully  experienced,  but 
that  most  will  be  undertaking  in  the  future,  is  user-controlled,  interactive 
decision  support  systems  (sometimes  called  the  "wild  what  if"). 

The  different  components  of  a  mature  decision  support  system  are 
shown  in  Exhibit  V-9.  Obviously,  this  is  an  application  that  will 
consume  many  computer  system  resources. 

One  aspect  that  should  terrify  data  processing  operations  management 
is  the  unpredictability  associated  with  a  "what  if"  capability  that  can 
access  a  detailed  operations  data  base. 

In  one  reported  case,  a  single  "what  if"  exercise  consumed  70%  of  the 
resources  of  an  Amdahl  V-7  system. 

The  exercise  was  not  considered  wasteful  of  computer  resources 
(within  the  boundaries  of  what  it  wished  to  accomplish)  and  was 
a  highly  important  piece  of  analysis  that  bore  rich  fruit  for  the 
company. 

However,  the  magnitude  of  the  request,  and  its  potential  impact 
on  operations  and  on  capacity  planning,  was  unforeseen. 
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EXHIBIT  V-9 


INTEGRATED  FINANCIAL  REPORTING 
AND  DECISION  SUPPORT  SYSTEMS 


COMPANY  OPERATIONS 
DATA  BASE 


EXTERNAL 
ECONOMETRIC 
DATA  BASE 


PLANNING 
DATA  BASE 


MODELING  LANGUAGE: 
FULL  RANGE  OF  EASE 
AND  POWER 

FULL  FINANCIAL 
STATISTICAL 
FUNCTIONS 

REPORT  GENERATOR 
(ENGLISH) 


FINANCIAL 
ANALYSIS 


MARKETING 
ANALYSIS 


BUDGETING 


FINANCIAL 
REPORTING 


SHORT-RANGE 
PLANNING 


LONG-RANGE 
PLANNING 
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This  kind  of  "what  if"  exercise  often  has  powerful  top  management 
sponsors  so  that,  even  where  the  analysis  is  pointless  or  uses  computer 
resources  very  inefficiently,  it  may  be  hard  to  oppose. 

Moreover,  it  is  this  kind  of  creative  use  of  computing  that  MIS 
departments  have  been  trying  to  encourage. 


The  capabilities  of  data  processing  installations  for  carrying  out  per- 
formance measurement  and  capacity  planning  will  increase  signif- 
icantly. This  will  be  brought  about  by  a  combination  of: 

Increased  knowledge  and  skills. 

Better  tools. 

The  difficulties  of  projecting  and  controlling  growth  may  also  increase 
significantly. 

The  importance  of  and  need  for  capacity  planning  will  grow  markedly. 
The  factors  causing  this  are  listed  in  Exhibit  V-IO. 


D. 


SUMMARY  OF  TRENDS 


In  general,  INPUT  sees  three  distinct  trends: 
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EXHIBIT  V-10 


FACTORS  INCREASING  IMPORTANCE 
OF  CAPACITY  PLANNING 


•  On-line  production  systems. 

•  Interactive  program  development. 

•  Lengthy  hardware  delivery  times. 

•  Budget  reductions,  general  economic  pressures. 

•  Large,  new,  high-visibility  applications. 

•  Major,  ongoing  enhancements  to  applications. 

•  Increased  choice  in  hardware  selection. 

•  Interactive  user  timesharing. 

•  Curvilinear  base  load  demand  increases. 

•  Distributed  data  processing. 

•  User  standalone  systems. 

•  Interactive  decision  support  systems. 
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RECOMMENDATIONS 


Capacity  planning  issues  may  only  occasionally  be  in  the  "you  bet  your 
company"  class;  however,  the  handling  (or  mishandling)  of  capacity  planning 
can  more  often  be  a  matter  of  "you  bet  your  job,"  or  at  the  least,  may  tarnish 
the  reputation  of  the  MIS  department. 

Mishandled  capacity  planning  can  result  in: 

Unpredicted  expenditures  for  more  equipment. 

Loss  of  corporate  business  (or  restricted  opportunities)  due  to 
bad  response  time  or  inability  to  expand  a  network. 

Delay  in  implementing  a  new  application  system. 

The  recommendations  given  below  are  grouped  into  four  categories. 

General,  or  strategic,  recommendations. 

Organizational  and  structural  recommendations. 

Recommendations  for  identifying  user  intentions  and  satisfaction. 

Tactical  and  methodological  recommendations. 
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A.       STRATEGIC  RECOMMENDATIONS 


•  Determine  the  appropriate  level  of  effort;  The  most  important  management 
decision  is  the  level  of  effort  that  should  be  devoted  to  performance 
management  and  capacity  planning.  There  are  two  principal  factors  here: 
resources  availability  and  organizational  acceptance. 

The  resource  constraint  is  an  obvious  one,  but  it  might  be  overlooked  or 
slighted.  On  the  most  basic  level,  a  small  organization  should  not  try  to 
do  all  the  things  that  a  large  company  might  attempt. 

Resource  constraints  can  be  useful  if  they  encourage  a  focused 
attack  on  problems  and  an  identification  of  high-priority  areas. 

Other  recommendations,  below,  will  describe  methods  of 
focusing  efforts. 

Potential  organizational  acceptance  is  a  less  obvious  issue,  but  can  be 
the  prime  determinant  of  the  program's  success.  To  be  quite  frank,  not 
all  organizations  are  good  candidates  for  a  thoroughgoing  program  of 
performance  measurement  and  capacity  planning.  This  can  be  due  to: 

Personalities  and  "politics." 

The  nature  of  the  business. 

The  level  of  organizational  advancement  of  the  MIS  department. 

The  dominant  personalities  in  some  businesses  are  extremely  mercurial 
and  resistant  to  planning.  Often  very  effective  because  of  their 
instincts  for  commercial  success,  they  nevertheless  offer  an  inhos- 
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pitable  climate  for  orderly,  long-range  planning.  The  best  approach  in 
this  environment  is  to  stay  flexible  and  always  have  a  considerable 
amount  of  excess  hardware. 

Similarly,  the  nature  of  some  businesses  is  quite  variable,  obeying  no 
predictable  pattern. 

Finally,  the  state  of  organizational  advancement  of  the  MIS  department 
itself  is  a  critical  determinant  of  what  can  be  expected  from  a  program 
of  performance  management  and  capacity  planning.  Exhibit  VI- 1  (taken 
in  simplified  form  from  INPUT'S  recent  study,  Improving  the  Productiv- 
ity of  Systems  and  Software  Implementation)  shows  the  general  stages 
of  MIS  development. 

A  quite  different  approach  to  performance  measurement  and 
capacity  planning  will  be  required  at  each  stage. 

At  the  "chaos"  stage,  for  example,  it  is  doubtful  that  a  very 
effective  program  can  be  devised.  Firefighting  and  a  reactive 
mode  will  be  the  most  that  can  be  expected. 

Train  data  processing  personnel;  The  state  of  knowledge  in  performance 
measurement  and  capacity  planning  is  in  flux  and  constantly  advancing;  most 
staffs  are  now  inadequately  trained.  There  are  many  sources  of  knowledge;  the 
right  ones  to  use  will  depend  on  the  organization's  current  level  of  knowledge, 
geographic  location,  goals,  budget,  etc.  Exhibit  VI-2  lists  and  evaluates  the 
main  sources. 

Educate  non-data  processing  management:  Managers  outside  MIS  must  under- 
stand the  importance  of  the  general  issues  involved.  This  is  necessary  for  two 
reasons: 

The  positive  contribution  and  support  they  and  their  staff  can  provide 
will  be  critical. 


-  97- 

©1981  by  INPUT.  Reproduction  Prohibited. 


INPUT 


EXHIBIT  Vl-l 


STAGES  OF  DATA  PROCESSING  DEVELOPMENT 


STAGE 

CHARACTERISTICS  AND  RELATIONSHIP 
TO  PRIOR  STAGES 

STAGE  0: 
CHAOS 

Self-descriptive. 

STAGE  1  : 
CONTROL 

The  constant  state  of  crisis  (created  by  the 
chaos  stage)  demands  efforts  to  bring  EDP 
under  control.  These  efforts  may  be  brutal, 
using  semi-arbitrary  mechanisms.  Rigid 

rp^ntra ligation       nno  mpthoH  wiHpIv  u^erl 

STAGE  2: 
QUALITY 

After  control  has  been  achieved,  the  mechanisms 
used  to  establish  control  are  modified  to  take 
into  account  the  demands  for  increasingly 
complex  systems,  more  reliable  and  more  at- 
tuned to  user  needs. 

STAGE  3: 
EFFICIENCY 

Quality  DP  systems  raise  interest  throughout 
the  organization  in  the  potential  for  DP  to 
increase  the  efficiency  and  effectiveness  of 
the  entire  organization.  Many  productivity 
initiatives  begin  in  the  quality  stage  and  are 
refined  and  expanded  in  the  efficiency  stage. 

STAGE  4: 

The  efficiency  stage  raises  expectations  further, 
since  user  needs  for  information  are  met  more 
predictably  and  information  has  critical  value 
alongside  the  other  factors  of  production. 

VALUE 

Data  processing  is  no  longer  a  separate,  isolated 
activity  but  participates  directly  in  the  main- 
stream of  corporate  activities  (as,  for  example, 
finance  does  now). 
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EXHIBIT  VI-2 


SOURCES  OF  INFORMATION 
ON  PERFORMANCE  MEASUREMENT 
AND  CAPACITY  PLANNING 


SOURCE 

COMMENT 

INTERNAL  DISCUSSIONS 
WITHIN    THE  COMPANY 

The  most  important  source  after  a 
minimum  level  of  knowledge  is  attained. 

INFORMAL  DISCUSSIONS 
WITH  COLLEAGUES  IN 
THE  COMPUTER 
COMMUNITY 

Especially  important  for  MIS  manage- 
ment; must  evaluate  sources  carefully. 

PERFORMANCE/PLANNING 
SOFTWARE  USER  GROUPS 

Can  be  on  the  leading  edge.  Danger: 
may  be  too  oriented  to  technical  details. 

HARDWARE  MANUFACTUR- 
ER USER  GROUPS 

Good,  especially  for  informal  contacts 
and  information  exchange  with  other 
users. 

rKUrbbb IUNAL  bUL.lt!  Itb 

Valuable,  if  not  too  theoretical  or 
rarified. 

VENDOR  BROCHURES  AND 
SALES  SEMINARS 

A  sometimes  overlooked  way  of  getting 
really  good  information  from  the  front 
lines  (they  have  seen  it  all) ;  quality 
and  depth  vary. 

SEMINARS 

Good,  if  a  particular  seminar  is  based 
on  extensive  exposure  to  the  field; 
some  can  be  too  general. 

SOFTWARE  DESCRIPTIVE 
REPORTS  (e.g.,  ICP, 
DATAPRO  SOFTWARE,  EDP 
PERFORMANCE  MANAGE- 
MENT HANDBOOK) 

Sketches  of  software  products,  very 
valuable  for  an  initial  review;  all  are 
basically  non-evaluative.  The  Performance 
Management  Handbook  provides  greater 
depth  and  a  good  overview  section. 
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The  capacity  planning  group  within  MIS  will  be  constantly  requesting 
planning  information  and  workload  data  from  user  departments.  Much 
greater  cooperation  will  be  obtained  if  it  is  founded  upon  informed 
consent. 

•  Direct  capacity  planning  from  the  top;  MIS  top  management  should  take  an 
active  role  in  developing  and  evaluating  the  performance  measurement  and 
capacity  planning  program. 

The  key  issues  should  not  be  considered  technical:  "War  is  too 
important  to  be  left  to  the  generals." 

A  well  run,  effective  capacity  planning  program  will  strike  to  the  heart 
of  an  enterprise's  affairs. 

MIS  management  will  often,  especially  at  the  beginning,  be  called  upon 
to  mediate  between  operations,  development,  and  user  groups. 

MIS  management  will  be  held  directly  accountable  for  failures  in 
capacity  planning. 

B,       ORGANIZATIONAL  RECOMMENDATIONS 

•  Treat  capacity  planning  as  a  generalized  function:  Performance  measurement 
and  capacity  planning  are  linked  in  that  one  is  a  foundation  for  the  other. 

However,  some  of  the  components  of  the  capacity  planning  process 
require  a  high  degree  of  knowledge  in  the  following  areas: 

Particular  application  systems. 
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The  organization's  goals  and  functions. 

Techniques  in  management  and  control. 

No  one  area  in  the  MIS  department,  least  of  all  the  systems  program- 
ming area,  will  be  able  to  produce  the  full  package  of  skills  required. 
Exhibit  VI-3  shows  a  quantification  of  the  types  of  knowledge  required 
in  these  widely  differing  areas. 

•  Set  up  a  task  force:  Because  performance  measurement  and  capacity  planning 
are  linked,  generalized  functions,  a  suitable  organization  structure  may  have 
to  be  specially  developed. 

The  typical  approach  (previously  illustrated  by  Exhibit  IV-2)  of  viewing 
the  program  as  primarily  technical  is  not  satisfactory,  except  as  an 
expedient  for  getting  started. 

A  better  approach  is  to  view  capacity  planning  as  a  "task  force" 
enterprise,  as  shown  in  Exhibit  VI-4,  with  individuals  specializing  by 
function,  but  working  as  a  unit. 

Personnel  need  not  be  full  time.  In  many  respects  it  is  more 
desirable  that  they  be  part  time  (but  at  least  one-third  time)  so 
that  there  is  additional  cross-fertilization.  A  part-time  role  is 
also  desirable  from  a  cost  and  backup  standpoint. 

The  task  force  could  report  to  the  director  of  planning,  since  the 
"higher"  attributes  of  capacity  planning  tie  into  planning  in  general,  as 
shown  in  Exhibit  VI-5. 

Smaller  organizations  could  have  the  capacity  planning  unit 
report  directly  to  the  MIS  director. 
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EXHIBIT  VI-4 


CAPACITY  PLANNING:    FUNCTIONAL  RELATIONSHIPS 


CAPACITY  PLANNING 
TASK  FORCE  CHAIRMAN 


TASK 
FORCE 
MEMBERS 


FORMAL 
LINKAGES 
TO: 


USER 
LIAISON(s) 


USER 
DEPART- 
MENTS 


DEVELOPMENT 
LIAISON(s) 


l 


MIS  DE- 
VELOPMENT 
CROUPS  (NEW 

SYSTEM 
DEVELOPMENT/ 
ENHANCE- 
MENTS) 


CAPACITY 
PLANNING 
TECHNICIAN(s) 


II 


fi 


PERFORMANCE 
MEASUREMENT 
AND 
IMPROVE- 
MENT 
^TECHNICIANS 


T 


OPERA- 
TIONS 
STAFF 
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RECOMMENDED  STRUCTURE 


VP-MIS 


DIRECTOR- 
OPERATIONS 


DIRECTOR- 
DEVELOPMENT 


DIRECTOR- 
PLANNING 


MANAGER- 
PERFORMANCE 
MEASUREMENT 

AND 
IMPROVEMENT 


STRATEGIC 
PLANNING 


CAPACITY 
PLANNING 
TASK 
FORCE 
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•  Ensure  intra-MIS  coordination:  MIS  management  should  ensure  that  the 
development  unit  of  MIS  keeps  operations  informed  of  all  its  activities  that 
would  affect  workload,  whether  new  systems  or  major  modifications. 

The  capacity  planning  task  force  can  provide  a  forum  for  this  activity. 

Whatever  life  cycle  development  methodology  is  employed,  there  should 
be  key  checkpoints  early  in  the  design  process  for  estimating  workload. 

Newer  methodologies,  such  as  BEST/ I  and  CRYSTAL  from  BGS 
Systems,  will  permit  design-based  metrics  to  be  converted  into 
workload  estimates. 

C.       IDENTIFYING  USER  INTENTIONS  AND  SATISFACTION 

•  Learn  users'  business  plans:  Show  top  management  that  MIS  needs  to  know 
user  plans  early  so  that  data  processing  implications  can  be  identified. 

The  MIS  function  must  learn  how  to  look  at  business  plans  from  the 
users'  viewpoints,  even  more  in  the  future  than  in  the  past. 

•  Identify  critical  response  times:  Not  all  respose  time  requirements  are  equally 
important.  Learn  which  ones  really  are  critical  from  a  dollars  and  cents,  as 
well  as  an  internal  politics,  standpoint;  and  learn  how  to  diplomatically  ensure 
that  this  point  of  view  becomes  part  of  the  decision-making  and  priority- 
setting  processes. 

•  Contain  "what  ifs":  Decision  support  systems  can  be  highly  useful  applications 
but  can  stop  data  processing  dead  in  the  water  if  they  get  out  of  control. 
Consider  the  following  alternatives  to  true  real-time  interactive  systems. 

Dedicated  minicomputers  with  an  extracted  data  base  of  company  data. 
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Overnight,  or  at  least  delayed,  turnaround. 
An  outside  timesharing  service. 

The  first  two  options  are  good  entry  approaches  to  set  the  terms  of 
supply;  the  latter  can  be  a  form  of  load-shedding. 

•  Provide  understandable  reports  to  users;  Performance  reporting  to  users 
should  be  straightforward  and  results-oriented.  Eliminate  data  processing 
jargon.  Think  about  what  the  user  is  really  interested  in  and  report  that. 

This  may  take  a  bit  of  initial  effort.  However,  in  many  cases  MIS 
management  will  find  itself  better  informed  also. 

•  Establish  an  MIS  "report  card";  MIS  will  regularly  report  its  perception  of 
planned  versus  actual  service  to  users.  Have  users  do  the  same.  The  MIS 
department  may  discover  some  communications  problems  at  the  least,  and 
perhaps  radically  different  perceptions  of  reality. 

D.       TACTICAL  RECOMMENDATIONS 

•  Plan  workload  shedding;  No  one  can  be  consistently  successful  in  planning. 
Therefore,  MIS  management  should  have  contingency  plans  for  dealing  with 
excess  workloads  (assuming  that  the  easy  way  out,  buying  more  equipment,  is 
not  available).  This  can  take  several  forms; 

Transfer  prime-time  batch  to  overnight. 

Lengthen  response  time,  using  the  following  criteria: 

Least  dollar  impact. 

) 
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Largest  job. 


Weakest  sponsor. 
Transfer  work  to  minicomputers. 
Use  commercial  timesharing. 

Send  work  out  of  house  (e.g.,  to  a  service  bureau  run  by  a  disaster 
recovery  service). 

•  Retain  deep  tactical  capacity  reserves:  In  order  to  keep  a  margin  for  the 
unexpected,  retain  some  slack  in  the  system.  The  credit  MIS  management 
thinks  it  is  getting  for  postponing  equipment  purchases  may  actually  be  quite 
small  (top  management  may  not  believe  its  claims).  Whatever  credit  is  gained 
by  running  a  tight  ship  will  be  nothing  compared  to  the  problems  caused  by  a 
capacity  crisis.  Examples  of  discreetly  established  reserves  include: 

Keeping  marginal  (sheddable)  batch  jobs  in  prime  time. 

Deemphasizing  routine  fine  tuning. 

Not  improving  service  beyond  levels  contracted  for. 

Having  stand-by  arrangements  with  hardware  monitor  consultants  in 
order  to  provide  extra  tuning. 

Targeting  discrete  functions  for  rapid  removal  to  minicomputers. 

•  Use  good  tools:  Make  sure  that  suitable  tools  are  available  (and  used). 

Everyone  should  consider  acquiring  a  general-purpose  software  monitor. 
Internally  processed  RMF  data  may  be  too  unwieldy. 
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Capacity  planning  tools  are  a  must  for  those  who  have  the  factual  base 
and  staff  to  support  them. 

Tools  will  continue  to  improve,  but  if  a  tool  is  worthwhile  now,  don't 
delay  action  in  the  hopes  that  a  better  one  will  come  along. 

Never  improve  service  unilaterally;  If  the  service  improvement  is  unneeded 
and  unexpected,  the  MIS  department  will  receive  little  credit. 

More  importantly,  the  higher  level  of  service  will  become  the  new 
"standard"  and  it  will  not  be  possible  to  slip  back  to  the  old  "adequate" 
service  level. 
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APPENDIX  A: 


INTERVIEW  SAMPLE  AND  METHODOLOGY 


•         Thirty-five  data  processing  installations  with  at  least  the  equivalent  of  a 
370/158  were  interviewed. 

The  profile  of  interviewed  installations  is  as  follows: 

The  processing  power  of  15  installations  was  less  than  a  3033. 

The  remaining  20  had  the  equivalent  of  a  3033  or  larger. 

The  smallest  configuration  was  somewhat  larger  in  power  than  a 
370/158-3. 

The  largest  installation  had  the  equivalent  of  five  3033s. 

Configurations  were  sized  according  to  CPU  ratings  in  thousands 
of  operations  per  second  (KOPS). 

Interviewees  ranged  in  position  and  responsibilities  from  senior  tech- 
nician to  executive  vice  president. 

Multiple  interviews  were  held  in  several  installations. 

Companies  across  a  wide  range  of  industry  groups  were  interviewed. 
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The  sample  was  nationwide. 


Nine  interviews  were  held  in  person  at  the  respondent's  installation;  the 
remainder  were  held  by  telephone. 

Eight  vendors  were  interviewed. 

They  included  leading  hardware,  software,  and  professional  services 
firms. 

Two  interviews  were  held  in  person  and  the  remainder  were  conducted 
by  telephone. 
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APPENDIX  B:  USER  QUESTIONNAIRE 

I.        Please  describe  what  your  organization  is  now  doing  in  each  of  the  following: 
COMPUTER  SYSTEMS  PERFORMANCE  MEASUREMENT 


COMPUTER  SYSTEM  OPTIMIZATION 


CAPACITY  PLANNING 
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So  that  I  have  an  overview  of  vour  operations,  tell  me  the  major  hardware 

and  software  you  are  using  now  (1981)  and  whether  there  have  been  any  sianificant 

changes  in  the  past  year  or  if  any  are  expected  next  year  (1982). 


BUDGET 

RESOURCES  1981  1982       ($  MILLION) 


Hardware  $ 
CPU  (model  //) 


Disks  (spindles) 
Terminals  (//) 


Software  $ 

Op.  Sys.     

(Data  Base)  DBMS    

Commun.  Monitor     

Other 


Personnel  (professional)      $ 

TOTAL  $ 


What  portion  of  the  above  are  under  the  direct  control  and  responsibility  of 
the  data  processing  department? 


o/ 
/o 


Describe  user  operated  and  distributed  data  processing  hardware. 


How  large  is  your  performance  measurement  and  capacity  planning  staff? 
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2d. 


What  rate  of  hardware  expansion  do  you  expect  over  the  next  five  years? 


3a.  How  important  is  computer  system  performance  measurement  to  the  manage- 
ment of  your  computer  operations?  (5  3  High,  3  =  Medium,  I  =  Low) 

•  Why? 

•  How  important  was  it  three  years  ago? 

•  Why? 

3b.  If  a  chargeback  system  is  used: 

•  At  what  level  does  control  exist  (job,  task,  etc.)? 


•  What  is  used  in  the  pricing  algorithm? 

CPU. 

I/O  usage  (EXCP). 

Class. 

Priority. 

Other. 

•  What  problems  do  you  have  and  how  do  you  plan  to  alleviate  them? 


3c.      Do  any  of  the  performance  measurement  software  tools: 

•  Use  graphics?  YES  (    )    NO  (  ) 

•  Operate  in  real  time?  YES  (    )    NO  (  ) 

•  Future  plans  for  graphics/real  time  tools?    YES  (    )    NO  (  ) 
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How  much  does  the  use  of  these  tools  increase  your: 


THROUGHPUT  RESPONSE  TIME 

•         Percent  improvement     

Variance  in  estimate  (+/-)     

Basis  for  estimate 


•         Additional  improvement 
possible  (%) 

Rationale 
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3f.       Are  you  aware  of  any  other  hardware  or  software  performance  measurement 

packages? 

PACKAGE  NAME        SOURCE  OF  KNOWLEDGE  OPINION 


3g.      What  is  your  biggest  unmet  need? 


4a.      How  do  you  plan  your  hardware  changes  (additions,  upgrades) now? 


•         What  is  the  general  process? 


•         Who  is  responsible? 


4b.      What  software  tools  do  you  use? 
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He.  Are  you  aware  of  any  software  packages  for  projecting  hardware  requirement? 

PACKAGE  NAME    SOURCE  OF  KNOWLEDGE  OPINION 


4d.  How  accurate  has  your  projecting  been,  i.e.,  how  close  (in  %)  are  your  6-12 

month  estimates?  Why?  Has  it  improved  over  time? 
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Is  this  margin  of  error  satisfactory?  Why?  What  have  you  done  to  improve 
it? 


We  would  like  to  explore  the  kinds  of  functions  that  would  be  most  useful 
to  you  in  projecting  your  hardware  requirements.  In  general,  what  level  of 
accuracy  would  you  like  to  have  the  further  you  get  into  the  future?  What 
level  are  you  now  attaining?  Express  as  a  percent  variation. 

VARIANCE  (%,  +  OR  -) 
TIME  INTO  FUTURE  TARGET  PRESENT 

1  mo.     

3  mos.     

6"  mos.    

I  yr.     

iy2yrs.     

2  yrs.     

3  yrs.     

5  yrs.     

Max.=   
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6a 


What  kinds  of  performance  measurement  software  tools  that  you  don't  have 
now  are  you  looking  to  vendors  to  supply? 


6b.      Do  these  exist  to  your  knowledge? 
YES(  )    N0(  ) 

•         If  yes,  what  are  they? 


•         If  no,  what  will  you  do  in  the  meantime? 
(    )  Nothing.     (    )  Do  it  yourself. 

7.  What  do  you  see  as  the  overall  trends  in  computer  systems  measurement  and 
capacity  planning  over  the  next  five  years? 

8.  What  impact  on  computer  systems  measurement  and  capacity  planning  do 
you  expect  from  such  specific  events  as: 

Falling  hardware  prices? 

Standalone,  user  operated  systems? 

Distributed  data  processing? 
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APPENDIX   C:   VENDOR  QUESTIONNAIRE 


APPENDIX  C:         VENDOR  QUESTIONNAIRE 


1.  How  effectively  do  data  processing  installations  use  current  performance 
measurement  and  capacity  planning  methodologies  and  products? 

2.  What  do  data  processing  installations  need  most? 

3.  How  do  you  see  current  products  changing? 

4.  Do  you  see  a  trend  toward  the  integration  of  different  products? 

5.  What  is  your  company  doing  to  make  performance  measurement  or  capacity 
planning  more  effective? 

6.  What  do  you  see  as  the  overall  trends  in  computer  systems  measurement  and 
capacity  planning  over  the  next  five  years? 

7.  What  impact  do  you  expect  from  such  specific  events  as: 

Falling  hardware  prices? 
Standalone,  user  operated  systems? 
Distributed  data  processing? 
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APPENDIX   D:  PERFORMANCE   MEASUREMENT  AND 

CAPACITY   PLANNING  TOOLS 


APPENDIX  D:         PERFORMANCE  MEASUREMENT  AND  CAPACITY 

PLANNING  TOOLS 


•  In  the  course  of  this  study,  information  on  a  number  of  performance 
measurement,  scheduling,  and  capacity  planning  software  tools  was 
accumulated. 

•  The  lists  shown  in  Exhibits  D-l  through  D-4  are  not  exhaustive  but  illustrate 
the  large  number  of  tools  available  (except  in  capacity  planning),  their  varying 
prices,  number  of  users,  and  ages.  No  single  reference  source  appears  to  have 
a  complete  list;  in  part,  this  is  because  new  products  are  constantly  appearing 
while  older,  less  successful  products  become  inactive. 
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